DLL calling is broken (wrong string and real values)

Reporter: time-killer-games  |  Status: open  |  Last Modified: March 14, 2019, 02:09:56 AM

So this used to work for me, but in my most recent test all values whether returned or passed as a parameter to a DLL, it is pretty random whether the numbers or strings show with the correct values or not.

DialogModule.zip

Run the test project attached above to see what I mean. This same DLL works just fine in GameMaker 8.1, GameMaker Studio 1.4, GameMaker Studio 2.2, Unity/C#, VB, VBScript, and JScript. Thus, this is an obvious bug with ENIGMA, and not the DLL itself. I can't reproduce this issue in any other engine or language. :-(

RobertBColton  
Seems to be a duplicate of #1461 and the regression was probably introduced by #1445.

Moved the declaration of window_handle out of WINDOWSmain.h since it's no longer included by Platforms/Win32/include.h as a result of the fact that it includes windows.h and causes identifier conflicts. It is kept only to hold implementation state for the Win32 platform.

So yeah, I tried moving the compatibility macro check into the source, but as #1461 clearly documents, that's currently broken.

NOBODY UNDO MY CHANGES WHICH WERE CORRECT THE MACRO SHOULD BE USABLE IN SOURCES

RobertBColton  

Also, there's very little history on the external functions, try taking a look at it and see if you can get something to work if it's nothing to do with the above.
https://github.com/enigma-dev/enigma-dev/commits/master/ENIGMAsystem/SHELL/Platforms/Win32/WINDOWSexternals.cpp
time-killer-games  

@RobertBColton on it as we speak!
time-killer-games  

@RobertBColton Here's the error that shows in the console btw

ERROR in some action of some event for object 0, instance id 100001: Attempting to access pointer variable as string

(I don't think the latest version of the file has this error)

time-killer-games  

@RobertBColton I just tried all previous versions and they all have the same problem, minus the above error.
RobertBColton  

That might be bad news for us, so if none of the above turns anything up, it could have to do with me adding pointer overloading for the var type. The var constructor may be confusing const char* and void* meaning it thinks all strings are actually pointers. Perhaps review f720a83 and or consult with @JoshDreamland because this is the last of my ideas on this subject.
time-killer-games  

@RobertBColton I would also like to point out that numbers are sometimes showing the wrong values as well. This I initially didn't entertain the possibility of, because I'm not used to it happening. But sure enough in some random cases completely wrong values for numbers are passed to arguments or returned by functions when printed or displayed somehow visually.
RobertBColton  

Are you still talking about DLLs only or do you now mean games in general, e.g scripts?
RobertBColton  

If you mean games in general, you might be getting affected by the #1448 GameData merge. If you'll notice, the Travis CI is also failing right now from it and we're trying to figure out why. There's also regressions in the compiler plus random epsilon errors.
time-killer-games  

@RobertBColton I'm just talking about calling DLL's
RobertBColton  

Ok, it could still be related to GameData anyway. Perhaps also review the history for DlgMod.
https://github.com/enigma-dev/enigma-dev/commits/master/ENIGMAsystem/SHELL/Widget_Systems/DlgMod

I'm out of ideas for now otherwise. The idea is to find where it was working and where it was that it stopped working. If you had enigma-dev locally I would tell you to be on the latest master branch and do git reset --hard HEAD~1 (go back and test 1 commit at a time) until you are on a commit where DlgMod starts working again.

time-killer-games  

@RobertBColton the history of DlgMod does not effect the DLL side. The DLL hasn't changed at all other than I added 3 functions since the time it was working. The test project does not use the DlgMod Widget System but rather uses scripts containing the external_* family of EDL functions to call the DLL within the project itself as aposed to using any Widget System whatsoever. The reason DlgMod doesn't work I am fully convinced is because of the same reason using the DLL in your EDL doesn't work; the external_* functions are broken somehow, I just wish we knew how or what the problem was and how to fix it.
RobertBColton  

Clone a copy of ENIGMA and do the git reset I mentioned until you find the commit where DlgMod works again, that's the way to find out.
time-killer-games  

Ok, will do! :)
time-killer-games  

@RobertBColton I've gone as far back as when sorlok fixed OS X and everything I tried since then has either produced the same issue or emake segfaults during build. I give up.
RobertBColton  

You were probably missing to make clean and make all -j4 on some of the commits, but sure. Just leave the issue, somebody can look when they have time. I'll post back if I think of or discover anything else.
time-killer-games  

Thank you Robert! Much appreciated!
RobertBColton  

Alright, so I finally downloaded the original project you provided. I didn't before because I didn't realize you made it that easy to just run it. Anyway, I wasn't actually able to reproduce any issues. I caught a few things though, one being the file filter on open dialogs seems to be corrupt. The extended open and save file dialogs seemed to be fine. Finally, it seems the 32 bit one doesn't work, which makes sense because I am building games 64 bit (and you can't load a 32 bit dll in a 64 bit process). I am back to thinking again that the issue is isolated to just you/your setup/PC and that perhaps an MSYS2 upgrade or just reinstalling MSYS2 is necessary.

Open File Dialog Filter Corrupted
Save File Dialog Filter

RobertBColton  

Now, after a little more investigating, I did find something that is in fact ENIGMA's fault sorta. The debug error messages are actually correct, but let me explain. They are coming from the var4 source file.
else show_error(string("Attempting to access ") + (type==0?"real":type==1?"string":"pointer")\

It is being triggered on the following line of the external source for Win32.

The specific thing that triggers them in your example is this:

external_call(external_define("DialogModule.dll", "window_get_caption_legacy", dll_cdecl, ty_string, 1, ty_real), window_handle());

Basically, because the compatibility setting is broke, and even if it wasn't, you might just have it enabled for window_handle() to return a var pointer in GMS compliance mode. You'll recall that I made specifically this change for you. But when you define your external call, you are saying that the type of the window_handle() is real, when it's not. When I added variant pointer overloading to ENIGMA, I added the type ty_pointer but because this may be incompatible with GMS, I want to look at some way of fixing this for extensions.

RobertBColton  

@time-killer-games So that leaves me with one question, what happens in GMSv1.4 if you pass window_handle() to a dll as ty_real? If that works, then ENIGMA needs to change something. If it doesn't, then we should close this and just fix #1461 and the compatibility setting.
time-killer-games  

@RobertBColton you just pointed out the issues I was talking about. Also notice the second screenshot you provided it is also using the title argument for the fname argument, this is also wrong.
time-killer-games  

@RobertBColton calling window_get_caption_legacy is a ty_real because it is meant for the GM678 compatibility setting. if you change window_get_caption_legacy to window_get_caption and ty_real to ty_string in the scripts that is meant for the GMStudio compatibility setting.
time-killer-games  

@RobertBColton You point out things that are wrong with the DLL calling and then you say everything on Enigma's end is fine. But that's not true. Those problems don't exist anywhere but in Enigma from my testing. And I've tested it in a lot of languages and software.
RobertBColton  

Actually, you have that backwards, but sure, I did not know that it was specifically the filter that was wrong and only in ENIGMA. I was just looking at the fact that 99% of the rest of it was working fine, so I was a little confused. Anyway, the window handle is just one part of the cast problem. I don't think that explains the file filter, so I wonder what is causing that?
time-killer-games  

The issue as I already stated is also present in the _ext() functions' fname argument. The fact any of the strings are working appears to be a fluke if you ask me. For instance, some of the input dialogs have a corrupt str argument for me. I'm sure if Josh tested this, he would also get a different result than the two of us.
time-killer-games  

@RobertBColton I updated the dll in the sample project to the latest version.
Please sign in to post comments, or you can view this issue on GitHub.