Pages: 1 2 3 »
  Print  
Author Topic: Rejoice  (Read 6112 times)
Offline (Male) Josh @ Dreamland
Posted on: February 18, 2010, 11:33:14 PM

Prince of all Goldfish
Developer
Location: Pittsburgh, PA, USA
Joined: Feb 2008
Posts: 2949

View Profile Email
Everything I wanted to work at this juncture works now. It's almost bittersweet; the teeth-grating hard part is seemingly over.
There is a problem, though. Typical student-made C lexers take upwards of minutes to parse the STL headers. I figured since I didn't need much of the data a compiler needs, I could do a lexless run-over to gather desired information, thus saving oodles of time. Well, it worked, but not well enough. On average processor load on my pretty average computer, parsing all the STL headers takes between one and two seconds. The latest result being 1.53662. Honestly, it's nothing to cry about, but this means I shouldn't just parse this stuff each compile, meaning Ism and I have to work something out to have LGM load ENIGMA as a DLL. This way, it will be able to retain such information.

I had typed up a report of what comes next on the school computers, but those have lately been failing to post here. I guess I'll summarize:

- As soon as I have access to a Windows box, I will reparse the STL headers on it, and then test windows.h to make sure it works (I have no doubt that it will). I also need to test the GL headers.
- Development will need to be carried out on said Windows device for a few cycles. This is because I assume said device (the one I have picked out) is 64 bit. Serprex had some problems with 64 bit machines in the past; I need to be sure they do not persist.
- Speaking of serp, I need his version of the system. He has been optimizing it for a while now. This will probably all be done inside tomorrow.
- Since the development of ENIGMA's parser, a new GM format has been released, along with a new EGM File format. ENIGMA needs to be versed in this new format. The parser also needs updated to use the CFile parser's list instead of the small list of built-ins I used to keep. This is a large step on the road to full extensibility.

And speaking of extensibility, with it come implications. Logically, I can't optimize for run speed, compile time, and output size all at once. So, here's what seems to be destined:
  • When testing your game, it will compile quite fast. The output size will be tremendous (approaching the size of GM output as the project grows). Speed will be "decent," which is great by GM standard. Error handling will be as follows, as debated long ago:
    • In regular run mode and release mode, errors will either kill you or they won't. They are not likely to be checked for.
    • In debug mode, thorough checking will be done that could cause slowdown
    • The jury is out on build mode.
  • When compiling your game for final release, the process will take from 10 seconds to a minute. This depends not so much on your computer as on your optimization settings. C++ comes with a set of native-related optimizations, ENIGMA's most notable optimizations are to be as follows:
    • Unused functions will not be included. This is possible thanks entirely to the CFile parser. [size optimization]
    • Unused local variables can be removed. [memory optimization] [possibly {semi-}manual]
    • Switch statements will be replaced with a type of hash map where possible. This will be another feat of mostly the C parser. [speed optimization]

Other optimizations become difficult to describe and list due to number and insignificance (they only add up in the long run). Not to mention I'm really tired.

Before I go though, I should elaborate on the implications of having removable systems.
Each of ENIGMA's systems is designed now to be optional. Since R3, when compile time was too high, these groups of related functions have been divided into individual source files. The problem with doing so is that many files in ENIGMA are highly interdependent. Many sources need a few members from instances, most often id, sprite_index, and mask_index. To give access to these variables without requiring all variables, R4 will require a super-parent, if you will, that contains only the most basic of locals. Other locals, like speed and direction, will be stored in a derived parent from which all other objects will derive. This will allow for all non-essential locals to be removed without major recompile.

*yawns* I need sleep today.
« Last Edit: February 18, 2010, 11:39:16 PM by Josh @ Dreamland » Logged
"That is the single most cryptic piece of code I have ever seen." -Master PobbleWobble
"I disapprove of what you say, but I will defend to the death your right to say it." -Evelyn Beatrice Hall, Friends of Voltaire
Offline (Male) Rusky
Reply #1 Posted on: February 19, 2010, 09:44:37 AM

Resident Troll
Joined: Feb 2008
Posts: 955
MSN Messenger - rpjohnst@gmail.com
View Profile WWW Email
How fast is "quite fast?" Also, how much can you separate the actual generated c++? One object per file? That would speed up compilation time quite a bit.

Also, your parsing the STL headers is rather redundant. You parse them to give EDL access, and then GCC parses them to compile the C++. You can't get the GCC-parsed version, but Clang has a much better architecture than GCC and when its C++ support is a little more mature you could probably get it to parse the headers for you and share the AST. Clang is also way faster and smaller than GCC while supporting the same things.
Logged
Offline (Male) Josh @ Dreamland
Reply #2 Posted on: February 19, 2010, 11:07:54 AM

Prince of all Goldfish
Developer
Location: Pittsburgh, PA, USA
Joined: Feb 2008
Posts: 2949

View Profile Email
Anymore i think you're sharing these things just to have something to share. "When its C++ support is a little more mature"? When the hell is that? Should I just wait for them to figure out what they're doing? Mine took a while, but it's done now (or as done as it needs to be). Let me know when Clang is more... mature.

As for "quite fast," it was used in lieu of a number. It meant "much faster than R3 as far as can presently be discerned." Though one object in each file would be nice (and doable) if LGM kept track of which objects were modified since last compile, I don't believe it does, so there isn't much point to putting them in separate files at the moment (I'll ask Ism about it next I see her, actually).
Logged
"That is the single most cryptic piece of code I have ever seen." -Master PobbleWobble
"I disapprove of what you say, but I will defend to the death your right to say it." -Evelyn Beatrice Hall, Friends of Voltaire
Offline (Unknown gender) luiscubal
Reply #3 Posted on: February 19, 2010, 12:37:08 PM
Member
Joined: Jun 2009
Posts: 452

View Profile Email
So you mean you currently merge the whole game into a single file. If so, then I don't see how can 1-2 seconds per compilation be that much.

Also, since so much of the compilation time is spent on the header parsing, rather than linking, I don't see how DLLs can help. You still need the STL/ENIGMA headers.
Logged
Offline (Unknown gender) The 11th plague of Egypt
Reply #4 Posted on: February 19, 2010, 04:43:56 PM
Member
Joined: Dec 2009
Posts: 276

View Profile
Wonderful news. This seems like a great piece of software engineering. Hell, I'm coding in C++ right now, but I can't quite understand much of your previous posts. :o

However, getting rid of all those unused functions and variables will cut down on the file size, which is something that got asked for a number of times.
Logged
Offline (Male) Rusky
Reply #5 Posted on: February 19, 2010, 06:10:17 PM

Resident Troll
Joined: Feb 2008
Posts: 955
MSN Messenger - rpjohnst@gmail.com
View Profile WWW Email
Clang has a production quality C compiler. It is written in C++ and is self-hosting. LLVM has a better optimizer than GCC. I say it's not ready with C++ yet because that's what they say. They're actively working on getting it to compile major C++ projects like Boost. I bring up Clang because you're having so much trouble parsing C++ yourself, and it has a C++ parser separated from its code generation, and good C++ support isn't far off.
Logged
Offline (Male) RetroX
Reply #6 Posted on: February 19, 2010, 06:52:41 PM

Master of all things Linux
Contributor
Location: US
Joined: Apr 2008
Posts: 1055
MSN Messenger - classixretrox@gmail.com
View Profile Email
As far as I know, removing unused functions, etc. has a G++ flag, someplace.  Either way, it's great that the parser does it.

And for the record, I have 64-bit Arch Linux installed, in the case that you need a test for a 64-bit Linux machine.
Logged
My Box: Phenom II 3.4GHz X4 | ASUS ATI RadeonHD 5770, 1GB GDDR5 RAM | 1x4GB DDR3 SRAM | Arch Linux, x86_64 (Cube) / Windows 7 x64 (Blob)
Quote from: Fede-lasse
Why do all the pro-Microsoft people have troll avatars? :(
Offline (Male) Josh @ Dreamland
Reply #7 Posted on: February 19, 2010, 09:20:49 PM

Prince of all Goldfish
Developer
Location: Pittsburgh, PA, USA
Joined: Feb 2008
Posts: 2949

View Profile Email
luis--
All the object data is in one file, but built-in functions are in many, many different files.

plague--
I'm hoping so.

Rusky--
It would be a great option if the team guaranteed its readiness. Until then, I'll stick with what I've worked on.

Retro--
I've not found such a flag, but I'd think there is one.
I will definitely need such a test done. The time will come for that.
Logged
"That is the single most cryptic piece of code I have ever seen." -Master PobbleWobble
"I disapprove of what you say, but I will defend to the death your right to say it." -Evelyn Beatrice Hall, Friends of Voltaire
Offline (Male) antidote
Reply #8 Posted on: February 19, 2010, 10:23:26 PM
Member
Location: Job Corps. <.<
Joined: Dec 2008
Posts: 27

View Profile Email
http://old.nabble.com/how-to-remove-unused-code--td15548009.html
Logged
Offline (Male) Rusky
Reply #9 Posted on: February 19, 2010, 10:26:29 PM

Resident Troll
Joined: Feb 2008
Posts: 955
MSN Messenger - rpjohnst@gmail.com
View Profile WWW Email
Of course, I wasn't suggesting moving to Clang right now.
Logged
Offline (Unknown gender) freezway
Reply #10 Posted on: February 19, 2010, 11:38:42 PM

Member
Joined: Dec 2009
Posts: 220

View Profile
wooohoooo cant wait for R4! i am really in need of game maker on linux!
Logged
if you drop a cat with buttered toast strapped to its back, which side lands down?
joshdreamland: our languages are based on the idea that it's going to end up FUBAR
/kick retep998
Offline (Unknown gender) Game_boy
Reply #11 Posted on: February 20, 2010, 07:44:11 AM
Member
Joined: Apr 2008
Posts: 228

View Profile
Quote from: Fede-lasse
Tremendous size? I thought ENIGMA was supposed to make them small, and, in particular, fast, and not fast by GM standards. 0_o

I think he means in testing mode, not when you compile it for release. Actually I think this way is a very good idea, since in GM you could test you game, make a change on one line and test it again in under a minute (whereas that's never been possible in C/C++ for large projects).

So when you need compile speed to be minimal (during testing), it compiles really fast and you don't need to care about size or performance since you're only testing a tiny fraction of gameplay.

And when it compiles for real you get almost the speed of actual C/C++, with a long compile time (but that only needs to be done once) and small executable size.

Is that what you meant Josh?

Logged
Offline (Male) Josh @ Dreamland
Reply #12 Posted on: February 20, 2010, 10:15:42 AM

Prince of all Goldfish
Developer
Location: Pittsburgh, PA, USA
Joined: Feb 2008
Posts: 2949

View Profile Email
Game_Boy--
Precisely.

Everyone else--
Listen to Game_Boy.

@"Perhaps you could implement a function set ... to make his/her application look out for errors"
The point of removing error checking is to save on multiple ifs each call. That's part of the reason GM is slow; the runner can never be recompiled to your needs. I would be no more happy to find that a user of my finished game had it throw an error on them (especially one that could be ignored) than to find that it had just acted funny or died (in the case that ignoring the error was inevitable).

@"you/the team has to work more on the error messages"
That's been a goal since the beginning.
« Last Edit: February 20, 2010, 10:19:07 AM by Josh @ Dreamland » Logged
"That is the single most cryptic piece of code I have ever seen." -Master PobbleWobble
"I disapprove of what you say, but I will defend to the death your right to say it." -Evelyn Beatrice Hall, Friends of Voltaire
Offline (Male) Rusky
Reply #13 Posted on: February 20, 2010, 12:16:48 PM

Resident Troll
Joined: Feb 2008
Posts: 955
MSN Messenger - rpjohnst@gmail.com
View Profile WWW Email
In GM you can tell it to log errors but not display them. That way when the game acts funny you still get the errors. I think that's a far better way of doing things than just ignoring errors- using compilation means a lot more errors get caught at compile time and the ones that are really runtime  don't cause enough slowdown to justify removing error checking.
Logged
Offline (Male) RetroX
Reply #14 Posted on: February 20, 2010, 12:21:59 PM

Master of all things Linux
Contributor
Location: US
Joined: Apr 2008
Posts: 1055
MSN Messenger - classixretrox@gmail.com
View Profile Email
http://old.nabble.com/how-to-remove-unused-code--td15548009.html
Yep, that's exactly what I was looking for.  This is what I usually use on compile:
g++ file.cpp -o file -Os -s
and now when I use this:
g++ file.cpp -o file -Os -s -ffunction-sections -fdata-sections -Wl,-gc-sections

It is slightly or significantly smaller depending on how many unused functions/variables/types/whatever that there are.
« Last Edit: February 20, 2010, 12:52:25 PM by RetroX » Logged
My Box: Phenom II 3.4GHz X4 | ASUS ATI RadeonHD 5770, 1GB GDDR5 RAM | 1x4GB DDR3 SRAM | Arch Linux, x86_64 (Cube) / Windows 7 x64 (Blob)
Quote from: Fede-lasse
Why do all the pro-Microsoft people have troll avatars? :(
Pages: 1 2 3 »
  Print