Debuging the .dll

[TheExDeus]

The crashes are killing me. Seeing as the problem is in the plugin (the .dll or the enigma.jar) I started trying to debug it. As the .dll is accessed from Java, then debugging it is non-trivial. What works for me is running LGM and then attaching gdb to the Java process. If it crashes after startup, then it could be as easy as looking up the PID in Task Manager and then typing:

Code: [Select]

gdb -p PIDSadly, for me LGM started crashing on startup. This means I have very little time to attach the debugger before the exception. Thankfully PowerShell can return the pid from the name and that can be then piped as the argument. So it looks like this:

Code: [Select]

gdb -p (get-process java |select -expand id)This finds the PID for java and attaches to that. So it is possible for me to do it the 2 seconds while LGM loads.
After that the exceptions thrown by the .dll are also shown there complete with backtrace if the are debugging symbols. To have them you must compile the .dll with the -g flag, but that is done by default in the master anyway.

Some things I noticed:
The JDI code is EXTREMALY unsafe. I understand it was written up to 5 years ago so Josh wasn't experienced (heck, I didn't even write C++ back then) and there was no C++11 features which are useful, but still, the code is a minefield. A lot of unsafe pointers, memory management done trough new/delete, a lot of unbound sprintf's which is an overflow waiting to happen and much more.
Like one of the reasons the plugin crashed was when a value of a template is invalid and so the char array is null. Then it calls std::string(null) and explodes, as that is a segfault. This apparently happens in numerous places I'm trying to fix it, but as I don't know the code that well I'm not sure what I can and cannot touch. Pointers are passed around without knowing if they are freed or not. Like now a segfault happens in the destructor of ~definition_template which has THREE for loops inside with "delete". This is a great way to segfault. Trying to sanitize all that is going to be a pain. I hope someone can help.

[TheExDeus]

Another thing I noticed is that LGM has a lot of breakpoints left inside, so gdb keeps stopping at them. When we pack the "release" version of LGM we should remove them.

Also,  LGM trows about 5 SIGSEGV's before it even loads. This could actually be it seeing on how many exceptions LGM actually shows (without crashing though). I thing LGM is also written in a very unsafe matter or at least null stuff is not checked. I know that LGM can crash if settings.ey is missing, if an extension misses something in a description, if LGM cannot find the compiler (that is probably the plugin though) and so on. All of those trown an exception when in reality that should be a warning at most.

[Goombert]

Harri, where are you getting this information? There is no concept of debugging symbols in Java. What you are likely referring to is the symbols that are in the native JDK implementation for your platform. You do not build Java programs with or without debugging symbols. It is all byte code and thus always has debugging symbols (sort of, but really a different concept). The only control you have over jars is whether to include the directory structure (since a jar is really a ZIP), what files to export, and whether to export the source code files along with the byte code (the byte code is in *.class files) generated from these sources. Obviously there's no need to include the source code generally speaking and it just increases the file size as does including the directory structure in the jar. When you build a Java program you convert it to byte code which is like cross-platform assembly kind of and it is fed into the JVM (Java Virtual Machine) that sits on top of the hardware. In .NET this is called the CLR or Common Language Runtime, .NET will actually take the generated byte code the first time you run the managed program and convert it to machine language (assembly) and store that for each subsequent execution. In Java this is somewhat analogous to JIT or Just In Time compile which converts the byte code to assembly "just in time". Anyway this is how Java solves the ABI compatibility problems, though I don't know about .NET because .NET programs make extensive use of versioned dll's.

Not to sound condescending or anything Harri I just want you to understand why that sort of doesn't really make any sense to suggest building LateralGM with debugging symbols, it has the debugging symbols, and that's how it can tell you the Java file and line number where an error occurred and the same is true for C#. That said what you suggest is likely true, someone should probably take the time to sit down and actually organize that part of the plugin if it's being obstinate about a missing compiler. I spent a lot of time putting the exception dialogs into LateralGM properly to avoid race conditions and infinite popup loops like you see in GM: Studio which you can thank me for later.

[TheExDeus]

In this topic I didn't mentioned debugging symbols in context of Java and I understand it was misleading to mention them in the previous topic. I was just referring to the fact that java.exe stops about 6 times while loading LGM. Look here: http://pastebin.com/GHegrqNA
This is running LGM WITHOUT ENIGMA. It is when I copied LGM outside in a totally different folder and run it. It of course runs fine and doesn't crash, but when I run it trough GDB it stopped 6 times. The first one was because of a breakpoint which means LGM still has some of them set in the version you uploaded (or maybe they are in Java.exe itself, but I doubt it), then the next 5 times it stopped with SIGSEGV. It didn't crash in a way that stopped the process as I could just type "c" and continue. Then these 6 stops later LGM was fully loaded and open. I could then use it and close it. Maybe this is normal for Java application, I am not sure. But it could show that LGM has some internal problems while loading too.

This topic is about the .dll though (parser). Because I just noted how I can debug problems in there too. If LGM crashes because of the .dll it will show up gdb and as the .dll does have debugging symbols I get a better error message. In java.exe case the backtrace is usually empty and I would need a debug version of java.exe to see them. But that wouldn't allow to pinpoint the problem down to LGM, because as you mentioned Java doesn't work that way. Usually LGM trows an exception with detailed path to problem, but it seems that LGM can also crash (like due to the plugin) without trowing anything.

[Rusky]

The JVM uses SIGSEGV intentionally, in order to implement its JIT. You can tell gdb to ignore a signal like this:

Code: [Select]

(gdb) handle SIGSEGV nostop
(gdb) handle SIGSEGV noprintWhen JNI code segfaults, the JVM aborts (possibly with a different signal, I don't recall for sure).

You also don't need to race the JVM with "gdb -p"- just run java from within gdb:

Code: [Select]

$ gdb --args java -jar lateralgm.jar
(gdb) r

[Goombert]

Yes Harri listen to Rusky, the debugging symbols/huffman codes are not my doing, it's Java's doing.

[TheExDeus]

By the way an example of problems the plugin has can be seen here: https://github.com/enigma-dev/enigma-dev/commit/6d6687824d1d6484c718b21d2ba6b31d80780d18
I fixed that null pointer nonsense and it reduced LGM crashes to almost zero on my laptop. More testing needed though. But the fact that a lot of code is so unsafe in the plugin is the reason why so many crashes happen.

[Goombert]

FYI my post over here was somewhat related to this:
http://enigma-dev.org/forums/index.php?topic=2584

[egofree]

I fixed that null pointer nonsense and it reduced LGM crashes to almost zero on my laptop.

Great news !  I am surprised : it seems we are using JNI, and i thought that the LateralGM plugin was using JNA.

[TheExDeus]

We are using JNA. The fixes I did was in the plugin code (JDI, the parser).

[egofree]

We are using JNA. The fixes I did was in the plugin code (JDI, the parser).

Indeed, JDI is not JNI 

[TheExDeus]

It seems that fixed bugs on my laptop, but on my PC it still crashes as usual.

[Goombert]

Are you guys ignoring my post about Breakpad? It's a really good idea, lets you debug without having debug symbols in the code using Google technology.

[TheExDeus]

It is useful if the users have the bugs as we can make some kind of autoreport system. But it is not useful for us or developers as we can just compile in debug mode.

[Goombert]

Yes I know what you mean but at the same time it increases the chances that they will report such issues, even more of them than we may be aware. It's like shining a flash light in the dark.