Linker crash

[TheExDeus]

I had a few linker crashes recently. I really cannot figure out why they happen. That only happens when I compile with Build>Compile. It works fine in Debug and Run modes. And the error also happens only when I enable one my extensions. It sometimes works, but often times it doesn't. I haven't yet checked what has changed on the extension side but I'm not sure that it is the problem. The LGM log is like this: http://pastebin.com/xFngiAMS
The error itself is very non-descriptive:

      0 [main] sh 7632 handle_exceptions: Exception: STATUS_ACCESS_VIOLATION
   2288 [main] sh 7632 open_stackdumpfile: Dumping stack trace to sh.exe.stackdump
   2288 [main] sh 7632 open_stackdumpfile: Dumping stack trace to sh.exe.stackdump

The crash dump is actually even less descriptive:

MSYS-1.0.12 Build:2012-07-05 14:56
Exception: STATUS_ACCESS_VIOLATION at eip=6E69572F
eax=00000000 ebx=6A626F65 ecx=FFFFFFFF edx=680A4C5C esi=69572F73 edi=776F646E
ebp=736A626F esp=0026B708 program=C:\ENIGMA\git\bin\sh.exe
cs=0023 ds=002B es=002B fs=0053 gs=002B ss=002B
Stack trace:
Frame     Function  Args
   4604 [main] sh 7632 handle_exceptions: Exception: STATUS_ACCESS_VIOLATION
   4903 [main] sh 7632 handle_exceptions: Error while dumping state (probably corrupted stack)

One thing I noticed is that the linker command is actually very long (more than 16k chars) and the Windows 8 limit is 8k. So I cannot even call the linker part from my cmd. But when I call the linker like this "g++ @commands.txt" where commands.txt has all of the string, I can get it to compile. No errors are thrown. Then I noticed ENIGMA uses sh.exe as the shell, probably just because of this limit. But when I call the same thing from sh.exe I still don't get the error. But when I use the "make", then it crashes: http://pastebin.com/ZUzmTcLp

So make.exe (or mingw32-make.exe if used instead) is the one that crashes. I looked on the net and few suggestions were given:
1) Disable AntiVirus or Windows Defender - did that, didn't work. This would also not explain why I can compile by just disabling an extension.
2) Run as Administrator - did that, didn't work. This too wouldn't make sense.

Later I will try installing new MinGW together with new Git and GCC and see if that helps. That will happen a week after the next one though.

Does anyone ever had this problem?

[Josh @ Dreamland]

If I had to guess, this is because Bash is capable of parsing any number of characters into an array of strings, while Windows is a worthless pile of slag. I don't know of a different way of conveying files to be linked to the linker, and those ways would be pretty barbaric, anyway.

There might be a way to switch Make's shell interpreter to Bash. Does running make from MSys-bash still give the problem, or only running it from cmd? The printout you gave looks like cmd.

There are ways to tell if you're in a bash environment from within the makefile, if you'd like to print errors.

Another solution would be to build static libs for each system before linking them. A third solution might be switching to cmake.

[TheExDeus]

Okay, so I found the problem. Finding it was a journey like no other. Basically it has nothing to do with the character limit I originally thought. Mostly because there is actually no cutoff when running make. The limit is when executing a command, but for that the command is a lot shorter:

mingw32-make.exe WORKDIR="C:/ProgramData/ENIGMA/" GMODE=Debug GRAPHICS=OpenGL3 AUDIO=None COLLISION=Precise WIDGETS=Win32 NETWORKING=None PLATFORM=Win32 CXXFLAGS="-std=c++11 -I../Additional/i686-w64-mingw32/include -g -DDEBUG_MODE" COMPILEPATH="Windows/Windows" EXTENSIONS=" Universal_System/Extensions/DateTime Universal_System/Extensions/Paths Universal_System/Extensions/NoVi Universal_System/Extensions/DataStructures Universal_System/Extensions/MotionPlanning Universal_System/Extensions/Alarms Universal_System/Extensions/BasicGUI Universal_System/Extensions/ParticleSystems Universal_System/Extensions/Timelines" OUTPUTNAME="C:/Users/Deus/AppData/Local/Temp/egm2873666830888467289.exe" eTCpath=""

Which is well within the limits of cmd. The larger linker part is the output which doesn't go trough cmd as far as I know. But CreateProcess (which I assume make uses) has an argument limit of 32,768 chars so it could be an issue as well later, but not here.

I tried Msys-bash - same thing - http://pastebin.com/SRC8YB4v . But then I noticed something - I had to install msys, I didn't have it. Together with ENIGMA we don't provide cygwin or msys - only mingw and git. When I install ENIGMA and run it historically the first thing I see is this:

Running make from `mingw32-make.exe'
Full command line: mingw32-make.exe Game WORKDIR="C:/ProgramData/ENIGMA/" GMODE=Run GRAPHICS=OpenGL1 AUDIO=OpenAL COLLISION=Precise WIDGETS=Win32 NETWORKING=None PLATFORM=Win32 CXXFLAGS="-std=c++11 -I../Additional/i686-w64-mingw32/include SHELL=cmd" COMPILEPATH="Windows/Windows" EXTENSIONS=" Universal_System/Extensions/DateTime Universal_System/Extensions/Paths Universal_System/Extensions/TouchSurface Universal_System/Extensions/AngelScript_Test Universal_System/Extensions/NoVi Universal_System/Extensions/DataStructures Universal_System/Extensions/MotionPlanning Universal_System/Extensions/Alarms Universal_System/Extensions/BasicGUI Universal_System/Extensions/ParticleSystems Universal_System/Extensions/Timelines" OUTPUTNAME="C:/Users/Deus/AppData/Local/Temp/egm9080526457944138510.exe" eTCpath=""
mingw32-make.exe -C ENIGMAsystem/SHELL
mingw32-make.exe[1]: Entering directory 'C:/enigma-dev-master/enigma-dev/ENIGMAsystem/SHELL'
process_begin: CreateProcess(NULL, uname -s, ...) failed.
mingw32-make.exe[1]: makefile:7: pipe: No error
mkdir.exe -p C:/ProgramData/ENIGMA/.eobjs/Windows/Windows/Run/
process_begin: CreateProcess(NULL, mkdir.exe -p C:/ProgramData/ENIGMA/.eobjs/Windows/Windows/Run/, ...) failed.
make (e=2): The system cannot find the file specified.

mingw32-make.exe[1]: *** No rule to make target 'C:/ProgramData/ENIGMA/.eobjs/Windows/Windows/Run/', needed by 'C:/ProgramData/ENIGMA/.eobjs/Windows/Windows/Run/SHELLmain.o'.  Stop.
mingw32-make.exe[1]: Leaving directory 'C:/enigma-dev-master/enigma-dev/ENIGMAsystem/SHELL'
makefile:12: recipe for target 'Game' failed
mingw32-make.exe: *** [Game] Error 2

I figured this was just a path issue, so I added the git/bin to PATH as well (as it comes with ENIGMA). This worked for years, but that apparently is the problem. In git/bin is sh.exe. Something I thought was necessary for mingw32-make to compile ENIGMA. Turns out it is the contrary - mingw32-make DOESN'T work with sh.exe. And sh.exe is the thing that is crashing here (original post is about it and the solution was here: http://www.cmake.org/Wiki/CMake_MinGW_Compiler_Issues). So I removed git/bin from PATH, but then I can't compile (the error I just posted) as it cannot find mkdir.exe for some reason. Then I went into an adventure where I found out that cmd has mkdir function and windows has mkdir program. So when I type mkdir.exe in cmd instead of running the mkdir.exe it actually creates a folder named ".exe" (as it calls the function, not the program). I couldn't figure out how to fix this, as even when calling in quotes (as suggested here: http://superuser.com/questions/856582/force-windows-use-exe-on-path-rather-than-cmd-exe-internal-command) it didn't work. Then in our own ENIGMA wiki I found that it mentions msys as mandatory, as Windows mkdir.exe is somehow inferior and doesn't allow something (http://enigma-dev.org/docs/Wiki/MinGW). So I ended up putting msys/bin in PATH for the mkdir function. But then we are back to square one, as msys/bin ALSO includes sh.exe... One way is to either delete it or rename it, but that is ugly. Another way is to specify "SHELL=cmd" to "mingw32-make" (http://stackoverflow.com/questions/24066265/force-mingw32-make-to-ignore-sh). Surprisingly we don't have a way to pass this flag to "make" in "Compilers/../gcc.ey". So I added it to CompilerSource. Now there is a makeflags: field which allows me to set SHELL=cmd and that now works.

Also note that windows doesn't have uname to tell which OS is running the makefile. But that is okay, as "process_begin: CreateProcess(NULL, uname -s, ...) failed." doesn't stop the makefile.

But then there is another problem. Apparently there is a syntax error in the batch for loop. So I get this:

echo "// GENERATED RESOURCE FILE FRONTEND" > C:/ProgramData/ENIGMA/.eobjs/Windows/Windows/Run/resources.rc
for res in Preprocessor_Environment_Editable/Resources.rc Widget_Systems/Win32/getstring.rc Widget_Systems/Win32/getlogin.rc Widget_Systems/Win32/showinfo.rc Widget_Systems/Win32/showmessageext.rc; do echo "#include \"$res\"" >> C:/ProgramData/ENIGMA/.eobjs/Windows/Windows/Run/resources.rc; done
res was unexpected at this time.
makefile:129: recipe for target 'C:/ProgramData/ENIGMA/.eobjs/Windows/Windows/Run/resources.res' failed
mingw32-make.exe[1]: *** [C:/ProgramData/ENIGMA/.eobjs/Windows/Windows/Run/resources.res] Error 255
mingw32-make.exe[1]: Leaving directory 'C:/enigma-dev-master/enigma-dev/ENIGMAsystem/SHELL'
makefile:12: recipe for target 'Game' failed
mingw32-make.exe: *** [Game] Error 2

Turns out the loop was written for sh.exe (or some other bash) as it didn't actually execute in cmd. There were several differences, but I fixed this too. Sadly now it doesn't run on sh.exe, but that shouldn't be an issue anyway. I don't know if you can write a shell for loop that runs both on cmd and sh.exe (or cygwin or something like that).

You can find the changes here: https://github.com/enigma-dev/enigma-dev/commit/642ba867d5f3d170c857752818278525d65a14f8

In the end nothing really has to be changed on user side, as msys is still not required. We can use the mkdir that comes with git, but we just need to force SHELL=cmd, which is now done.
Took a while to figure this out.