Win32 platform has incomplete Unicode support

Reporter: EnigmaBot  |  Status: open  |  Last Modified: October 06, 2017, 02:21:36 AM

(Posted by time-killer-games on the ENIGMA forums)

try putting this in the room creation code of and empty project with one room to see an example of this:

show_message("тест");

it appears none of ENIGMA's strings will support such characters.

RobertBColton  
draw_string should be able to handle that text with the appropriate font set. What does GameMaker do when you call show_message with that parameter? Either way, ENIGMA should support UTF-8 in those dialogs just for simplicity. Probably a matter of accepting a wstring for the parameter.
JoshDreamland  

TKG and I have discussed this offline—ENIGMA and GCC are both handling the unicode admirably (this is mostly a facet of UTF-8's ingenious design and LGM's stewardship of it for user code). The problem is that Win32 does not. We're using the default implementation of WinAPI macros, or explicitly calling the A version rather than the W version, so our UTF-8 is being interpreted as whatever Microsoft maps the non-ASCII byte range to by default. (Latin-1?)

To "fix" this, we'd need to pore over all Win32 calls and swap in the "W" versions to make UTF-8 work properly on Windows. TKG has confirmed it works fine on Linux.

RobertBColton  

Yeah please see here:
https://msdn.microsoft.com/en-us/library/windows/desktop/ff381407(v=vs.85).aspx

Internally, the ANSI version translates the string to Unicode. The Windows headers also define a macro that resolves to the Unicode version when the preprocessor symbol UNICODE is defined or the ANSI version otherwise.

That should turn on UTF-8 for all of Windows without changing much of our code.

Be careful: Some headers use the preprocessor symbol UNICODE, others use _UNICODE with an underscore prefix. Always define both symbols. Visual C++ sets them both by default when you create a new project.

RobertBColton  

Ok, I got it working in #1077 only for show_message right now. This is an annoying issue and is going to require that we go back and change all calls to the Win32 API in places we care about UTF-8 support. For now at least, the most basic show_message will support UTF-8 and the helper functions are defined to convert the strings easily. Feel free to take my changes when they are merged and run wild with that helper to update the rest of the Win32 platform if you want.

win32utf8

Relevant commits:
bc3dbb3
efe16dc

NOTE: The original version we committed had a memory leak according to Josh so we had to update it again later, this is the safer version which will actually be used:
e107274

Feel free to just copy and paste those changes if you can't wait until #1077 is merged. Leave this ticket open until we solve the problem for all of Win32 please.

time-killer-games  

I find this pretty related to the topic so I thought I'd post this here.

If you try to run an ENIGMA game when it is in a directory that contains any special UTF-8 characters, you will get this error on startup:

bug

What you see is an empty game project with 1 room, and no code. Like the other UTF-8 bugs, this is a Windows-only problem.

Notice the folder it's in has special characters, in this case the folder is called "тест".

Please sign in to post comments, or you can view this issue on GitHub.