ENIGMA Forums

General fluff => General ENIGMA => Topic started by: egofree on June 25, 2013, 11:00:37 AM

Title: Overuse of the CPU ?
Post by: egofree on June 25, 2013, 11:00:37 AM
As i am developing a game, i am trying to optimize the code. I did some tests, but i found that even with the most simple project possible (one sprite, one object, a room with one object, no code, no special events) the CPU use is 13 % !  As i use a rather powerful computer with an Intel Core i7 CPU and with 16 GB of memory this seems to me a little bit overkill. I tried the same project with GM 8.1 and the CPU use is not significant. What do you think ?
Title: Re: Overuse of the CPU ?
Post by: Josh @ Dreamland on June 25, 2013, 11:09:03 AM
What's your GL support look like? It's possible something is being done software side which should not be. Also, what is the room speed of your game? High room speeds mean high CPU usage, just by virtue of doing something many times per second.
Title: Re: Overuse of the CPU ?
Post by: egofree on June 25, 2013, 11:40:02 AM
I am using the latest drivers from NVIDIA for my GTX 660 card. As far as i understand the Open GL drivers are included in these drivers. The room speed is the default one : 30. I am running Windows 7 64 bits.
I tried to edit the ENIGMA settings and to use DirectX 10.0 instead, but the compilation crashes. I've DirectX 11.0 installed on my computer.
Title: Re: Overuse of the CPU ?
Post by: Goombert on June 25, 2013, 12:14:34 PM
egofree, theres a reason for that, we don't have DirectX ^_^
Title: Re: Overuse of the CPU ?
Post by: egofree on June 25, 2013, 12:53:50 PM
egofree, theres a reason for that, we don't have DirectX ^_^

Yes, i know there is no DirectX on linux  :). But still a lot users are using Windows, and i am afraid that if you need a powerful computer on Windows to run ENIGMA games, it will hinder its success  :( I am still using Windows because it's convenient if you want to run mainstream games, but i hate the new Windows 8 with Modern UI and i really hope linux will get more and more success in the future.
Title: Re: Overuse of the CPU ?
Post by: TheExDeus on June 25, 2013, 01:27:10 PM
Empty games usually use more CPU. Just like the most intensive part of SC2 was the menu (where there was a sphere rendered). The same here. Your GPU has nothing to do so the CPU boost to the max. It shouldn't really happen at 30FPS though. I previously found a problem that having the room_speed set over the max CPU/GPU can handle, then CPU will spike to 100% (for the one core it runs on), you lower the FPS by 1 (like 61 to 60) and it falls to 0%. We need a more intelligent system which would run in the background and monitor and change the performance. Although VSYNC and adaptive vsync (which we should implement, it's the same function like we have now, but with -1 set instead) it usually doesn't happen.

And I just tested with one animated sprite with one object and in one room with default 30fps. I get 0-1% CPU load. I also have i7, 16gigs of RAM and 660TI. So it could be something else on your side. Is something else using the CPU? As I said, it seems to use a lot of CPU when it can't catch up. So if you can't easily render 30FPS because of some massive processes, then the game could just up in CPU usage. It seems like a devilish circle, but for now I don't know what can be changed. For me that happens when I render over 300FPS (or more) in the 1mil cube demo.

edit: Also I though we had a memory leak because ram usage kept increasing from about 18mb on startup to about 20mb after about 1 minute. Then it stabilized though. I don't think we cache anything so it seems weird.
Title: Re: Overuse of the CPU ?
Post by: egofree on June 25, 2013, 02:28:03 PM
Is something else using the CPU?

No, in the tasks manager, i see nothing unusual for others processes. Well, i don't know what is the problem for the moment. We will see. It would be interesting to know if others windows users have the same problem, or i am the only one.
Title: Re: Overuse of the CPU ?
Post by: egofree on June 25, 2013, 02:29:45 PM
egofree, theres a reason for that, we don't have DirectX ^_^

I heard an interesting rumor about migrating to MinGW64 and support of DirectX. Does anybody have some news about this rumor ?  :)
Title: Re: Overuse of the CPU ?
Post by: Goombert on July 03, 2013, 12:30:17 AM
Yes, good luck getting it to link correctly, I tried and tried, I can only get the DirectX10 headers included, but I can not manage to link the libraries, knock yourself out. :P
Title: Re: Overuse of the CPU ?
Post by: TheExDeus on July 03, 2013, 08:42:12 AM
I doubt DirectX would change anything. It just happens that for some hardware/software combinations stuff like that happens. I doubt the CPU use would increase even if the room wasn't empty. Just like when people cried that GM games are 2mb even when just having 1 empty room, but when you make the game it usually grows to only 4mb. So it's hard to tell why it shows 13% CPU load, but it's possible the load wouldn't change with a basic game.
Maybe now something has changed? Like downloaded newer drivers or installed/uninstalled something and it shows a different load now?
Title: Re: Overuse of the CPU ?
Post by: egofree on July 03, 2013, 11:00:33 AM
Maybe now something has changed? Like downloaded newer drivers or installed/uninstalled something and it shows a different load now?

I've installed the latest drivers from Nvidia today (320.49), and the result is the same.
Title: Re: Overuse of the CPU ?
Post by: egofree on July 03, 2013, 11:01:59 AM
I doubt DirectX would change anything.

I don't know, but i heard that the support of Open GL on Windows is not good. That's why i was thinking about DirectX.
Title: Re: Overuse of the CPU ?
Post by: TheExDeus on July 03, 2013, 02:31:59 PM
Well anything is possible, but CPU load shouldn't be changed by the graphics system. Maybe some more detailed profiling is possible to see what exactly the game is doing on the CPU. Looking at this thread http://social.msdn.microsoft.com/Forums/vstudio/en-US/ef38c900-f2e2-4ae7-8e03-2a3941805aa3/profiling-what-code-causes-high-cpu-usage even the most basic timer profiling could do the trick. You could check which function executes the longest and thus see why it shows that CPU usage at 13%. In that case it is possible that the some GL function (like GL swap buffers or something) cause a stall which takes a larger amount of time and thus shows up as higher CPU usage (while in reality CPU was just waiting).

Try turning off the background in room, if that doesn't show anything then set automatic redrawing to false. I don't think "Sleep on minimize" thing is implemented, but if it is then try that as well.
Title: Re: Overuse of the CPU ?
Post by: Goombert on July 03, 2013, 03:05:07 PM
Harri, forthevins update timer takes into account wherever you are rendering things or not, and tries to pick a suitable CPU usage.
Title: Re: Overuse of the CPU ?
Post by: TheExDeus on July 03, 2013, 04:49:31 PM
But the timer works between frames, I am thinking something goes wrong during the rendering (or as nothing is really rendered, then the buffer flipping). Still, profiling can be done and checked why it consumes CPU like that. I have almost identical PC to him and I don't have that problem. I still think he should also try some games or example to check if CPU usage actually increases. Maybe because of some compatibility problems the 13% will be the default and additional computation will not really increase that.
Title: Re: Overuse of the CPU ?
Post by: Goombert on July 03, 2013, 05:44:08 PM
Deus, also, I am planning a profiler to be integrated with debug that will output things such as ram and vram and cpu usage >:
Title: Re: Overuse of the CPU ?
Post by: egofree on July 04, 2013, 04:31:20 AM

Try turning off the background in room, if that doesn't show anything then set automatic redrawing to false. I don't think "Sleep on minimize" thing is implemented, but if it is then try that as well.

It doesn't change anything. I tried also with OpenGL 3.0 but it doesn't change the result. By the way, it's not possible to make an executable with Open 3.0, i've error messages. I've found an Open GL profiler : http://www.gremedy.com/free_license.php. Here is the result :

(http://nsa33.casimages.com/img/2013/07/04/130704113120708586.jpg)

(http://nsa34.casimages.com/img/2013/07/04/130704113249204430.jpg)

(http://nsa34.casimages.com/img/2013/07/04/13070411340312609.jpg)

I don't know if it's useful for you.
Title: Re: Overuse of the CPU ?
Post by: TheExDeus on July 04, 2013, 05:56:33 AM
Well first, GL1 doesn't work for me but GL3 does. So that is also weird.
Second, was this for empty game?
And can you give the information about profiling - like the time and cpu usage inside the debugger. The information you provided is what GL functions it called, but sadly it doesn't give information on how long they took.
I can render an empty room with just 1 object which draw fps with 0% CPU load. Even when I render 999FPS. When I set it to 9999FPS then I get 2400 actual FPS and cpu load is 16%. This is on a laptop with an ATI card and 2.5Ghz I3. When I check CPU load with profiler I get 3-12% and it varies all the time. In the task manager I still get 0% though.
Do you use any anti-virus?

Thanks about the profiler link. Didn't know it's freeware now. It is very good profiler.

edit: And Robert, do you plan to remake all the drawing functions OGL3? I see that it still uses immediate mode.
Title: Re: Overuse of the CPU ?
Post by: egofree on July 04, 2013, 02:53:22 PM
Second, was this for empty game?

Yes.

Do you use any anti-virus?

Yes, but i tried already to disable it, but it doesn't change anything. With gDEBugger, i tried also to see if it's possible to see CPU use for each function, but i don't think it's possible. Finally i tried an c/c++ profiler on Windows : very sleepy (http://www.codersnotes.com/sleepy (http://www.codersnotes.com/sleepy)). Here is the result :

First screen :

(http://nsa34.casimages.com/img/2013/07/04/130704094935943658.jpg)

Second screen, it's the breakdown of the thread which uses almost all CPU power :

(http://nsa33.casimages.com/img/2013/07/04/130704094159458293.jpg)

And my task manager (to prove i am not lying  :D ) :

(http://nsa34.casimages.com/img/2013/07/04/130704095406331559.jpg)
Title: Re: Overuse of the CPU ?
Post by: TheExDeus on July 04, 2013, 03:16:37 PM
It seems the very cool sleep function isn't that cool after all. Try setting room speed to 999 (which should disable sleep) and see what you get (and check if you get stable 999, if not then set it lower so it's stable).
Here is mine: http://imageshack.us/a/img59/1533/52h1.png
So I clearly don't have it. It is weird that sleep eats so much CPU for you though. Maybe forthevin knows more.
Title: Re: Overuse of the CPU ?
Post by: egofree on July 04, 2013, 04:54:57 PM
It seems the very cool sleep function isn't that cool after all. Try setting room speed to 999 (which should disable sleep) and see what you get (and check if you get stable 999, if not then set it lower so it's stable).

Interesting. I tried many different values, and here is the result:

Room speed between 1 and 59 -> CPU 13 %
Room speed between 60 and 999 -> CPU 25 %
Title: Re: Overuse of the CPU ?
Post by: Josh @ Dreamland on July 04, 2013, 05:43:22 PM
It uses 13% CPU at 1 FPS?
Title: Re: Overuse of the CPU ?
Post by: egofree on July 05, 2013, 02:00:37 AM
It uses 13% CPU at 1 FPS?

Yes, i tested also with with room speed 1, and CPU use is 13%. In an 'empty' project, it seems there is only two possible values for CPU use on my computer : 13 and 25 %.
Title: Re: Overuse of the CPU ?
Post by: TheExDeus on July 05, 2013, 02:01:31 AM
Josh, by the looks of the profiling the Sleep() function is the one eating the CPU. So if it sleeps for 59/60 then it will eat as much or even more CPU. Maybe I am wrong though as I can't find a topic on the internet about Sleep() using much CPU. Here is an interesting article on how to make it use certain amount of CPU - http://www.codeproject.com/Articles/28870/Better-Way-to-Sleep-Control-Execution-and-Limit-CP . But I don't think we should limit a game to run at certain %. Could be a cool optional feature though for creating programs that are meant to run in background.

25% is 1 core and you will not get more with a single threaded application like an enigma game. Though that means you exceeded stable FPS. Try drawing fps (draw_text(10,10,string(fps)) to see what you actually get. If you set it to 100, then you get 100? And max CPU load (25%)? And when set to 400 you also get 400, but with max cpu load (25%)?
Title: Re: Overuse of the CPU ?
Post by: egofree on July 05, 2013, 02:37:42 PM
25% is 1 core and you will not get more with a single threaded application like an enigma game. Though that means you exceeded stable FPS. Try drawing fps (draw_text(10,10,string(fps)) to see what you actually get. If you set it to 100, then you get 100? And max CPU load (25%)? And when set to 400 you also get 400, but with max cpu load (25%)?

Room speed   FPS    CPU load
-------------------------------------
30                   30        14
60                   60        25
100                 60        25
400                 60        25
Title: Re: Overuse of the CPU ?
Post by: Josh @ Dreamland on July 05, 2013, 03:18:40 PM
I just couldn't believe that Sleep() implementation is so stupid. Operating systems are supposed to love Sleep, because they can all but completely ignore a sleeping thread until its condition is met. Does Notepad use 13% CPU on your machine? :P
Title: Re: Overuse of the CPU ?
Post by: egofree on July 06, 2013, 04:46:48 AM
Does Notepad use 13% CPU on your machine? :P

Only Notepad++ does, but Notepad is ok !  :P  :)
Title: Re: Overuse of the CPU ?
Post by: Goombert on July 06, 2013, 11:36:56 PM
Here I got the answer Josh, Windows sucks, there you go have a nice fucking day. As far as those deprecated GL functions go, much of the OpenGL3 graphics system is still using software rendering for some things, just about the only thing I've changed is using Vertex Buffers for models.
Title: Re: Overuse of the CPU ?
Post by: TheExDeus on July 07, 2013, 05:57:22 AM
But as far as I can tell he is the only Windows user we have who has this problem. It's weird, but still shows that it could be something specific to him. ego, have you tried on another PC? And if you sometime in the future reinstall Windows then try again. And did you actually install the OS or was it preinstalled with the PC?

edit: And what is this TestOpenGL.exe you are running all these tests? Is it really an empty game? I guess you would notice if it wasn't...
Title: Re: Overuse of the CPU ?
Post by: egofree on July 07, 2013, 06:56:07 AM
But as far as I can tell he is the only Windows user we have who has this problem. It's weird, but still shows that it could be something specific to him. ego, have you tried on another PC? And if you sometime in the future reinstall Windows then try again. And did you actually install the OS or was it preinstalled with the PC?

edit: And what is this TestOpenGL.exe you are running all these tests? Is it really an empty game? I guess you would notice if it wasn't...

I've totally re-installed my PC (Alienware X51) several weeks ago, because i know it had a lot of crap on it and i've installed also an anti-virus. I tried to install as much as possible open-source software.
I've got another mini-PC, a Dell Studio Hybrid, which is several years old, with a dual boot (Vista and Linux mint). I tried the empty project on Vista : CPU load 50 % !!! I know the computer is not very powerful, but we are talking about an empty project. I analysed the exe with anti-virus and it doesn't seem to find any viruses :https://www.virustotal.com/en/file/a4aa8fe98dfcb0a08a71ada1e4f96ae193d8eec06428cd8110cd6a421e7eb7fb/analysis/1373196738/ (https://www.virustotal.com/en/file/a4aa8fe98dfcb0a08a71ada1e4f96ae193d8eec06428cd8110cd6a421e7eb7fb/analysis/1373196738/). I tried also to install ENIGMA on this mini-PC and i created an empty project, but the result is the same. Java is up-to-date.

To create an 'empty' project, i just do this :  right-click sprite, create a new one, and choose an image, then right-click object, create a new one and choose the sprite, and finally right click room, create a new one add the object to the room. In any case, you can download there the exe of the empty project : http://www35.zippyshare.com/v/24257123/file.html (http://www35.zippyshare.com/v/24257123/file.html)

I know i already told you, but in any case, thanks for your support, TheExDeus, i really appreciate.
Title: Re: Overuse of the CPU ?
Post by: TheExDeus on July 07, 2013, 12:32:20 PM
Well now we are getting somewhere. On that particular .exe I also get 13-14% CPU load (also pretty constant). But some questions:
1) Can you upload the gmk?
2) How did you compile it? When I launch I see that no console pops up. I don't think that has been fixed in ENIGMA for Windows, so I should still see the console. To remove the console you have to change the makefile to make it a different project (I once did that, but now cannot remember how I did it).
3) Can you try this: https://dl.dropboxusercontent.com/u/21117924/testGame.exe
It is basically the same you did. Only I used a different sprite I had lying around. When you run it you should see a console popping up along the main window. On it I get 0-1% CPU load. I somehow think you could be using some old or buggy ENIGMA. The one I used to compile here is 1 day old (on my main PC I didn't actually have a working version, so I reinstalled everything from scratch yesterday).
Title: Re: Overuse of the CPU ?
Post by: Goombert on July 07, 2013, 03:12:28 PM
ego, you don't need to do that you can just hit run on a game with absolutely no resources, we have built that into the engine ^_^
Title: Re: Overuse of the CPU ?
Post by: egofree on July 08, 2013, 02:33:04 AM
Quote from: TheExDeus
1) Can you upload the gmk?

Here is the egm file : http://www45.zippyshare.com/v/7127148/file.html

Quote from: TheExDeus
2) How did you compile it? When I launch I see that no console pops up. I don't think that has been fixed in ENIGMA for Windows, so I should still see the console. To remove the console you have to change the makefile to make it a different project (I once did that, but now cannot remember how I did it).

I've installed the Windows installer version and installed the updated jar from Robert (http://enigma-dev.org/forums/index.php?topic=1245.0). I didn't make any others modifications. Then i just compile the project with Build -> Compile. I tried to get the latest version from the repository, but i didn't succeed : http://enigma-dev.org/forums/index.php?topic=1320.msg13376#msg13376

Quote from: TheExDeus
3) Can you try this: https://dl.dropboxusercontent.com/u/21117924/testGame.exe
It is basically the same you did. Only I used a different sprite I had lying around. When you run it you should see a console popping up along the main window. On it I get 0-1% CPU load. I somehow think you could be using some old or buggy ENIGMA. The one I used to compile here is 1 day old (on my main PC I didn't actually have a working version, so I reinstalled everything from scratch yesterday).

I get 2-3 % load with your exe, and the console is visible.
Title: Re: Overuse of the CPU ?
Post by: TheExDeus on July 08, 2013, 06:05:15 AM
Even 2-3% is a little high, but it's in the ballpark where it should be.
I just compiled your egm and I can run it with 1% CPU use like normally. So I am sure you have some buggy ENIGMA installed. In the topic you mentioned that Robert's version worked and so you probably didn't try the one I mentioned. Maybe try that (it should work when you have two versions extracted. Just extract it to another folder). I don't know about Robert's version, but the one I posted downloads newest stuff from GIT.

We should REALLY fix this installing BS. Previously we didn't have working installer, but now we have several which "supposedly" works, but apparently differ in content. Which lead to this topic and took 3 pages to figure out. So I propose:
1) Pack everything just like Poly did, so on first run it would download everything from git. I also think it would be nice if it showed if newer commit is available and propose to download that as well.
2) Pack the newest LGM with it. The poly version has old plugin and old LGM which creates hoops you must jump trough to get it all working.
3) Rename the LGM correctly (or change ENIGMA.exe accordingly), so you don't have to rename it every time you want to install it.

Of course the problem could also be with his MinGW. But for now I think it is the ENIGMA.
Title: Re: Overuse of the CPU ?
Post by: forthevin on July 27, 2013, 03:45:49 AM
Hey egofree, I have pushed a change which may fix the issue. Could you test with the new version and see if it solves the issue?
Title: Re: Overuse of the CPU ?
Post by: TheExDeus on July 27, 2013, 04:08:07 AM
I think the issue was his ENIGMA. When I sent him .exe compiled with newest version, then he didn't have high CPU usage.
Title: Re: Overuse of the CPU ?
Post by: forthevin on July 27, 2013, 06:00:32 AM
Ah, ok, that's probably it. Then the change will probably help avoid any future issues.