Introduction
I wanted to make a topic about changes in ENIGMA. Forum has been quiet for a while and nothing much has been going on. But I'm still here and still working on ENIGMA. So some of the stuff done in the past few months will be described here.
TL;DR
At least 40 bug fixes. At least 297 new functions (about 262 for BasicGUI extension). Texture atlas which can increase FPS up 24x. A much more mature GUI extension. 64bit compilation for windows. ENIGMA not dead.
Texture atlas
It took years, but ENIGMA is officially as powerful as any decent engine in the beginning of last decade.
We have texture atlas (or as GM call them - texture pages). It basically packs several sprites into one texture, so there isn't any expensive texture changes. This is extremely useful for 2D games and GUI/UI/HUD, as they usually involve a lot of 2D textures. This is less important for 3D games.
Previously a code like this was the worse case scenario for ENIGMA:
int i = 0;
repeat (10000){
int spr = spr_0;
switch (i){
case 0: spr = spr_0; break;
case 1: spr = spr_1; break;
}
draw_sprite(spr,-1,random(room_width),20+random(room_height)-20);
++i;
if (i>1) i = 0;
}
draw_set_font(font_0);
draw_text(10,10,string(fps));
draw_text(10,30,"The quick brown fox jumps over the lazy dog");Here 10k sprites are drawn, but they change image one after the other. So in reality there are only two images - 5k times one is drawn and 5k times the other. Here one sprite is a green pentagon and the other sprite is a red one. Here is a screenshot:
You can see I only get 23FPS here. Reason for that can be seen here:
We can see in one frame we call 70k OpenGL functions. We actually call 10k draw calls (one for each sprite) as well as 1 for the text. So it's 10001 draw calls to draw the frame. You can also the there are 4 textures in memory and one of them is visible in the image (the red pentagon).
To use texture atlas I added a few new functions. Right now it is useable at runtime (unlike GM which can only be used in the IDE), so this is the code required:
texture_page = texture_atlas_create();
texture_atlas_pack_begin(texture_page);
texture_atlas_pack_sprite(texture_page, spr_0);
texture_atlas_pack_sprite(texture_page, spr_1);
texture_atlas_pack_font(texture_page, font_0);
texture_atlas_pack_font(texture_page, -1);
texture_atlas_pack_end(texture_page);That is easy right? We create a texture atlas page, then add two sprites and two fonts (including the default -1 one) and then _end(). In the _end() it actually does the packing. It is very efficient and uses the Josh's rectpack which we already used for fonts. Specifying a size for the atlas texture is optional and it is calculated automatically to be the smallest power-of-two texture it can be. After calling this code the texture looks like this:
You can see we only need 12 OpenGL function calls per frame and there is actually only one draw call. There is only one texture as the rest were merged and destroyed. The packed texture is on the right. Now the fonts and sprites can be used as normal and nothing changes, so existing code works fine. You can also see that the font characters are packed per character, not per font texture, so spaces between sprites are packed with fonts. Unlike GM which is quite wasteful (can be seen
here).
This is the output for the example after running the texture_atlas code:
We get 560FPS instead of 23FPS. That is 24.3x speed up (2430%). This works in DX9, GL3 and GL1 graphics systems and you can pack sprites, fonts and backgrounds (so all textural resources in ENIGMA).
TODO:1)This only works in code and is not implemented in LGM. I like it that way, but it would be useful for LGM to pack textures too so I wouldn't have to do it at runtime (which is extremely fast, but could still be a slowdown with thousands of sprites). Allowing to do it at runtime does seem important, as you can now even pack sprites you loaded externally. GM doesn't allow that.
2) There could be a few more options added, like padding. The system also doesn't check if a texture is already packed (if you try to pack the same sprite twice the result is undefined now). And lastly we could allow the same texture to be packed multiple times. So you could optimized the atlas at runtime. Like if you had a desert world it could be packed together with UI. Then the next ice world could also be packed with UI so you can draw as much as possible with one draw call.
BasicGUI improvements
Those who don't know BasicGUI is an extension I'm making for ENIGMA. It adds GUI stuff like windows and widgets. They are not meant to be external of the main window, so they are all drawn inside. Useful for game UI or editor UI. The extension includes windows, buttons, labels, toggles, scrollbars, sliders as well as skins, groups and parenting. It is inspired by Unity system, but it is a little verbose because of EDL limitations. So right now the extensions consists of 262 functions.
As simple example:
This shows windows, group toggles (basically radio buttons), sliders, buttons with child labels (the button with the "Lena" picture), scrollbars and labels (the larger "Lena" picture). I get 6600FPS here because I also packed everything into a single texture. This would draw in one draw call if it weren't for a stencil buffer I used that will be described later.
Another example is the node editor I'm working on.
Everything you see there is drawn using the BasicGUI extension (excluding the connecting curves, which are drawn using draw_bezier_cubic_color() function). I get 2430FPS and I also use atlas here, so here is the texture:
It takes a lot more OpenGL function calls here because I use surfaces and stencil buffers for cutting stuff off outside BGUI window. If I would hide those windows and not draw surfaces, I would be able to draw the whole thing in
one draw call. It is a lot more cooler in action, so if I make a video of it I will post it here. The BasicGUI extension is graphics system agnostic - it uses only generic drawing functions and should work for all graphics systems that implement them. Now they are GL1, GL3 and DX9.
TODO:1) Add textbox widget.
64bit for Windows
I say "for Windows", because I think Linux and MacOS had this working for some time now. But on Windows there had to be some few fixes for this to work. There is actually performance reasons to compile in 64bits, because it can increase fps. Like here (press to enlarge):
The only difference is that one is compiled in 32bits and the other in 64. The difference is not large (about 4%), but it is still 100fps. 64bit of course uses a little more memory. 32bit uses 29.8mb or ram while 64bit uses 32.1mb.
Here is the atlas test:
Here we also get almost 100fps or about 14%. 32bit uses 50.9mb while 64bit uses 57mb.
All in all this is great. 64bit's of course also mean we can use more than 2GB of ram. Most 2D games don't care about the 2GB limit and most 3D games rarely hit it too (AAA games of course do). I'm dealing with a lot of data not connected with games and so for me the possibility to use more than 2GB is very useful.
TODO:1) Compile the rest of libraries to 64bit and create a new windows installer which has these libraries. Right now I only compiled the libffi, so I can compile a game. I need to compile OpenAL, ALURE, Box2D and some others.
Stencil buffer
I added some simple stencil buffer functions. They are primarily used in the GUI system so that windows cut off content that is outside of it. It's like using surfaces to do it, but without the additional VRAM. A simple example:
repeat (5000){
int spr = spr_0;
draw_sprite(spr,-1,random(room_width),20+random(room_height)-20);
}
d3d_stencil_start_mask();
draw_circle(room_width/2,room_height/2,room_height/2,false);
d3d_stencil_use_mask();
repeat (5000){
int spr = spr_1;
draw_sprite(spr,-1,random(room_width),20+random(room_height)-20);
}
draw_set_font(font_0);
draw_text(10,10,string(fps));
draw_text(10,30,"The quick brown fox jumps over the lazy dog");
d3d_stencil_end_mask();And the output:
What happens here is that I draw 5000 red sprites. Then I start the stencil mask and draw a circle on it. Then I use the mask to draw the rest 5000 green sprites and the text. The green sprites and the text is limited to the circle I drew. So it is masking which pixels can be written to. This works in GL1 and GL3.
TODO:1) The functions need to be changed so we can use several values in the stencil mask.
Fixes
-Fixed normal matrix.
commit-Model_floor and model_wall fixes (changes necessary because of normal matrix change).
commit-fixed sprite_create_from_screen and background_create_from_screen.
commit-Direction is now rounded. Fixes a problem where vspeed = 5, made direction = 269 instead of 270.
commit issue-Added the maximize button if window resizing is enabled.
commit-Fixes string_width(" "). Previously if the string only consisted of spaces, then string_width() returned 0.
commit-Fixed definition of draw_set_line_pattern.
commit-Fixed double define for draw_spline_part with wrong arguments.
commit-Removed glsl_program_bind_frag_data from header. It was never implemented and I cannot even find out what it is (it's not a GM function either).
commit-Added definitions for font_get_glyph_texture_left/top/right/bottom. They were implemented, but not defined.
commit-Remove matrix_ functions and d3d_transform_vertex which were not implemented.
commit-Remove export_include_file, discard_include_file and include_file_location as they were not implemented.
commit-Added empty functions for d3d_set_software_vertex_processing in GL. Software processing is idiotic anyway, but D3D supports it, and I need to make a stub until we make platform specific functions easier to implement.
commit-Fixed d3d_model_part_draw() definitions - they missed vertex_start argument.
commit-Removed display_get_orientation as it hasn't been implemented (we don't even support devices with orientation right now).
commit-Removed joystick_map_button and joystick_map_axis from PFjoystick.h, as they are not implemented in Windows, but they are added in Linux. As they are defined in LINUXjoystick.h, then I guess they still should work on Linux.
commit-Fixed sound_get_pan and sound_get_volume return's.
commit-d3d_draw_torus is now defined properly.
commit-Removed duplicate draw_mandelbrot define.
commit-Fixed room_get_name. For all resource get_name function the default return value was "<undefined>", but room_get_name is implemented differently. It isn't declared in IDE_EDIT like the rest. And when given incorrect room index it would just crash in non-debug version, so I made it return "<undefined>" in this case instead.
commit-Surfaces are now using unordered_map instead of regular arrays of pointers. This was causing a memory issue (Dr. Memory crashed on "new surface"). This is more C++ way anyway.
commit-Fixed bug in the new surface creation, where I actually increment surface_max count even though the id itself was reused.
commit-Fixed memory leak in graphics_copy_texture.
commit-graphics_copy_texture now correctly crops the image.
commit-Fixed the font packer so it would return power-of-two textures.
commit-Some small optimizations in GSbackground. It's very possible compiler on O3 did that anyway.
commit-Removed some warning from GL3textures.cpp.
commit-Removed GSEnable.h and corresponding .cpp files. They are not used anywhere and they implement functions that are already in d3d_ category.
commit-Fixed .obj loading. There was an error that when you load an .obj with normal values, but without texture coordinate values, then the normals were all messed up. This is now fixed in GL1 and GL3.
commit-Fixed d3d_set_fill_mode not drawing in GL3.
commit-Added NOCHANGEDIR flag to dialogs. Previously the get_open_filename and get_save_filename dialogs changed working directory. It was messing with some other stuff and as we cannot change working directory with any built-in function right now, then I don't think this was intended. So I added a flag that forbids changing of the directory.
commit-Disabling zwrite now works correctly in GL3.
commit-Widgets now compile for 64bit. Some code in win32 widgets needed to be changed so it would compile for 64bit.
commit-Windows widget rc files don't show warning anymore. They were because of the manifest.xml include, but I added include guards in the files themselves just as additional measure.
commit-Fixed the for loop in makefile that dealt with windows resources (rc files) as it didn't run on cmd. It was made for sh.exe or something like that, but we can't use it.
commit-It is not possible to pass flags to make. This is required to set SHELL=cmd in windows. It fixes problems described here:
http://enigma-dev.org/forums/index.php?topic=2488.0 commit-Recpack had a limit of 255 rectangles it could pack. That is way too low for a texture atlas which can have thousands. So I fixed that. Texture atlas also had to be changed for this to work. Now the limit is an "max unsigned int".
commit-For 64bit's we need to pass a flag to windres. I added this to the e-yaml's and compiler, so it is possible to pass it.
commit-The compiler is now compiled with -O3 which does make the parsing faster.
commit-Built-in shader is now a C++11 raw literal. This makes it easier to maintain and copy, as we don't need those damn quotes and newlines.
commitAdded or implemented
-Added room_first and room_last.
commit-Quadratic bezier curve now uses width given by draw_set_curve_width().
commit-Implemented font_get_glyph_left/top/right/bottom. They were fined but not implemented.
commit-Implemented triangle_area, which was defined, but not implemented.
commit-Implemented sound_get_pan and sound_get_volume in OpenAL, both of which were defined.
commit-Implemented display_get_gui_height and display_get_gui_width.
commit-Implemented date_get_week and date_inc_week. Apparently I missed it when I wrote the thing in 2011. 4 years later, it's in (though not ISO).
commit-Implemented mp_grid_clear_cell which was clearly missing.
commit-Implemented sprite_get/set_bbox_mode. This was also done in the .dll.
commit-Implemented d3d_light_set_ambient and d3d_light_set_specularity in GL1.
commit-Added graphics_copy_texture() which is required for texture atlases.
commit-Added graphics_copy_texture_part. This is also needed for texture atlas, as I need a way to copy only part of the source texture in fonts.
commit-Added functions d3d_stencil_start_mask, d3d_stencil_use_mask and d3d_stencil_end_mask which allow easy use of stencil masking.
commit-Added d3d_transform_set_array and d3d_projection_set_array which take pointer to an array to set the 16 values.
commit-Added d3d_transform_add_array, so you can add an array as well as set it. Same with d3d_projection_add_array.
commit-Added at least 262 BasicGUI extension functions.
-Added 7 texture atlas functions.
And many more smaller fixes here and there. All of this is in this branch:
https://github.com/enigma-dev/enigma-dev/tree/GL3.3NormalMatrixNext for ENIGMA
I plan to work more on the stuff here as well as other interesting and useful features. But sadly I don't know how much I can do alone. There are not active developers right now besides me. One soar point is the parser. It was mostly written and rewritten by Josh, but he is not that interested in ENIGMA anymore. So there is no one that can actually fix the many bugs and issues we have with the parser. I propose changes to the EDL to make the parser a lot simpler, like getting rid of the dynamically added variables. All variables local to an instance will have to be declared as "local". This is actually a small change that would break little, but could fix a lot of problems we have now with std::maps crashing in the parser. I don't need - or even want - GML compatibility. Or GM compatibility in general. We don't need people porting stuff from GM to ENIGMA. We need people making stuff on ENIGMA. Seeing as GM is slowly dying anyway, I don't think we need to follow them. I would like if EDL was much more closer to c++ and it might as well be a little more strict.
Another problem is the IDE. LGM on Windows is extremely unstable. I have to restart it 5 times before the Run button actually runs, otherwise it just freezes. egofree mentioned he might be free at the end of this month and look into it. I heard others have continued to work on NaturalGM, but it doesn't have a commit since last year, so it does seem dead. Together with RadialGM and other IDE's.
So I might end up making a branch from ENIGMA. In it I would try replacing the parser with something much more simple (as I would basically need instance system to work and that is it) and the IDE. That of course is still a lot of work which I don't have resources for to do alone. I will probably make a separate topic about.
