ENIGMA Forums

General fluff => General ENIGMA => Topic started by: TheExDeus on April 29, 2014, 03:16:02 pm

Title: Variable performance
Post by: TheExDeus on April 29, 2014, 03:16:02 pm
Found a GM topic here: http://gmc.yoyogames.com/index.php?showtopic=604444
Some users basically tried to measure several different variable read/write performances. So I though I could compare to ENIGMA. Sadly I don't have YYC so I can't test it, and comparing to regular GM wouldn't really be fair. So this is YYC from the topic:
(http://s22.postimg.org/lxvp1x1up/Windows_YYC.png)
So here is ENIGMA:
(http://i.imgur.com/Z7u4Yd6.png)
It does seems we are faster at few and slower at others. Comparing the two isn't correct though, as the YYC result wasn't done on my PC. But I do believe some of the iterative methods we have now are indeed slower than YYC could have (as theirs could be coded better, as it is few years newer as well). Just for comparison here is what happens when the "var" is replaced by "int" (so when we declare the type):
(http://i.imgur.com/N3T84dL.png)
We can see that all of the ones which previously used "var" and were slow are now A LOT faster. Some went from 3000us to 0us (to it was not even measurable) while others went from 115000us to 7000us (16x speedup).
Many others didn't change because I can't override things like built-in variables to use int. So I changed only the ones having specific "var" declaration. I also changed the string, so you can actually see which ones were changed. So look for the ones having "int" in the name.
So while there are things we should improve engine wise, we still have the ability to define variables that give a massive speedup. And I encourage people to use them.

This topic was created in the "series" of the other topics involving variables.
Title: Re: Variable performance
Post by: Darkstar2 on April 29, 2014, 04:09:24 pm
Interesting, now in real world, does this matter ?  People judge the games by the overall frame rate.  Even under worst scenarios with tons of variables and such would probably not reduce game speed by something noticeable most people would perceive.

Also the YYC would probably yield better result compared to non YYC, as this is CPU operations, and the YYC benefits CPU and not GPU intensive games.  Also I would think that the memory speed can have an impact too, depending on people's configuration but again, we are talking very precise measurements that would not be noticeable in most cases, if at all.

I don't have YYC to test this :)

Title: Re: Variable performance
Post by: TheExDeus on April 29, 2014, 06:23:30 pm
I know that YYC yields better results than non-YYC. It's actually in the topic I linked to and I mentioned that it's more proper to compare ENIGMA to YYC.
Title: Re: Variable performance
Post by: Darkstar2 on April 29, 2014, 07:09:20 pm
I know that YYC yields better results than non-YYC. It's actually in the topic I linked to and I mentioned that it's more proper to compare ENIGMA to YYC.

I wonder how the YYC would compare to ENIGMA ?
Do you think ENIGMA's would still be faster than their YYC ?  If going by their track record I would think ENIGMA has the edge on speed :D
Title: Re: Variable performance
Post by: TheExDeus on May 01, 2014, 11:02:51 am
Please, read the OP. The graphs ARE showing YYC and ENIGMA.
Title: Re: Variable performance
Post by: Darkstar2 on May 01, 2014, 12:20:40 pm
It would be fair to test all those possibilities on the same system.  The OP obviously does not include ENIGMA.  It would be good for someone here who has the YYC to make these comparisons.  Fortunately, I don't have YYC, I know someone here who does perhaps he can volunteer :D



Title: Re: Variable performance
Post by: Ideka on May 01, 2014, 07:16:42 pm
I'm sorry, what's YYC? Google says it's Youngstown Yatch Club or something.
Title: Re: Variable performance
Post by: Darkstar2 on May 01, 2014, 07:28:29 pm
I'm sorry, what's YYC? Google says it's Youngstown Yatch Club or something.

It's basically a contraption that turns your projects into a jumbled messed up version of C++ and compiles it.  It's called YoYo Compiler.  It's included with the $800 MASTER COLLECTION, and it costs a whopping $300 if you don't have master (for GMS Pro users).....  My advice save your money !  :D

Games CPU intensive benefit from YYC.

This however would be irrelevant for ENIGMA since it already compiles and does basically what GMS should have done all along ! :P
Title: Re: Variable performance
Post by: TheExDeus on May 02, 2014, 09:44:37 am
Quote
The OP obviously does not include ENIGMA.
The original post has 3 images. The first one is YYC (but not my PC) and the last two is ENIGMA (on my PC).
Title: Re: Variable performance
Post by: FroggestSpirit on May 02, 2014, 11:04:42 pm
I'm working on switching my known variables over in Project Collidascope. This leaves me with a few questions...
Can arrays be declared as specific types? if so, how?
Can we make data structures like in C++?
I think there should be highlighting for the variable types, or bold it like when you type var
Title: Re: Variable performance
Post by: TheExDeus on May 03, 2014, 07:46:10 am
Quote
Can arrays be declared as specific types? if so, how?
Can we make data structures like in C++?
The answer for now I think is that you cannot. It's a parser bug as it tries to use GM arrays instead (which are actually functions). Like:
Code: [Select]
some_array[10] = 0;;
some_array[0] = 5;
Actually turns into:
Code: [Select]
some_array(10) = 0;
some_array(0) = 5;
While real C++ arrays need to be:
Code: [Select]
local int some_array[11];
some_array[10] = 0;
some_array[0] = 5;
So compiler errors out. For now there is nothing you can do to go around this bug (as far as I know). The second problem with structs is the same. You cannot use them until the parser is done. Josh keeps "working on it", but hasn't given any real indication on when he plans to finish or what problems does he have now.

Quote
I think there should be highlighting for the variable types, or bold it like when you type var
This is an LGM issue. I think some of the keywords are now hard coded which they shouldn't be. I think keyword lists (both functions and otherwise) be external, so it would allow easy additions of these kinds of things.
Title: Re: Variable performance
Post by: Goombert on May 03, 2014, 01:12:38 pm
They are hard coded when ENIGMA is not plugged in, when ENIGMA is plugged in keywords for functions but not types are added by ENIGMA.
Title: Re: Variable performance
Post by: Josh @ Dreamland on May 17, 2014, 08:41:27 am
I'm glad we actually have a benchmarking tool. This can be fixed with a little reflection, and depending on the user's mood, a little memory. Throughout ENIGMA's past, I've hemmed and hawed about how to treat variables referenced in with blocks. The current behavior is to look them up in the same manner as ID-referenced variables. The other behavior was to scope them into everything, so they exist and are located in the same place in every object, and therefore are trivial to access.

A third behavior I have not attempted is using virtual tables to store the variable access methods. This would make arbitraryId.myvariable into getter(arbitraryId)->get_myvariable(). This moves the burden from a switch statement based on object index to a simple virtual call. We're probably already incurring a by-pointer function call, so this could only mean savings.

The simplest way to take care of the whole "var" fiasco is with some static analysis of the code. This cannot be done with the current compiler.

Note: the behavior you are seeing, ie, GM being faster at setting a single var index, is due to the fact that GM's variables are always a map. So nothing special is constructed when you need to assign to a var. It's not that assigning to a var is expensive—constructing one is. There might be a way to improve that by enforcing a single var implementation, and moving the contents of var4_lua.cpp into var4.cpp. This would remove the need to do the allocation tricks I've done, and expose all code to the optimizer. Another quick trick might be replacing std::map with a faster map type; you could try hash_map.

Please do a benchmark with variant instead of var.
Title: Re: Variable performance
Post by: TheExDeus on May 17, 2014, 10:01:26 am
Here is result by replacing every "var" with "variant":
(http://i.imgur.com/Z7UF8zs.png)
I actually thought that it is the same thing. But variant is actually faster in almost every case.

The benchmark should actually be written better, but I didn't want to modify it so we could compare to GM. We could make it so we can compare with different ENIGMA types though.
Title: Re: Variable performance
Post by: Josh @ Dreamland on May 17, 2014, 11:37:38 am
Variant is scalar (unless you count the fact that string is an array itself). Var uses some memory magic to throw together a Lua table. Construction time is pretty horrifying, but it has a good use time.

Those global assign times are horrifying. I'm not sure how to deal with those, but since they are literally worse than assigning instance variables, I do have one obvious solution. :P

Don't get me wrong; the with assign times are abysmal, but those can stand for some obvious improvement.
Title: Re: Variable performance
Post by: Goombert on May 05, 2019, 08:09:07 pm
We finally got around to this and the var is now as fast as variant in benchmarks.
https://github.com/enigma-dev/enigma-dev/issues/1617
https://github.com/enigma-dev/enigma-dev/pull/1676
Title: Re: Variable performance
Post by: Josh @ Dreamland on May 06, 2019, 10:10:26 pm
The bigger takeaway is that variant is frequently as fast as double. Not always; depends on C++ optimizations, right now. When it's not as fast as double, it's still pretty fast.