ENIGMA Development Environment
Website is in read-only mode due to a recent attack.

Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.

Messages - Josh @ Dreamland

Announcements / We can't decide
« on: November 03, 2011, 12:46:26 PM »
The ENIGMA project has hit a snag recently. Basically, our collective ideologies have become tangled.
It looks something like this:


Ism and I are both sorting issues with our projects. Ism is dealing with Java's ill-equipped generics, trying to structure LGM to be more extensible for future releases. As far as you need to be concerned, this is so we can add our Definitions and Overworld resources, as well as other potential resources down the road.

As for myself, right now I am dealing with a small issue regarding a dynamic type that implements implicit accessors and setters for generic data, which has presented a number of "how should I"s, but in the big picture, my concern is for the progression of ENIGMA's parser.

One point is certain: I need to rewrite or do serious work on the C parser. ENIGMA is presently ill-equipped to make distinctions between the following sample lines:


This has led to issues in interpreting some GM6 code unless extraneous parentheses are added, which is unacceptable to newcomers, as well as to issues implementing some of C++'s more desirable features, such as complicated ternary expressions. This is because ENIGMA needs information from C.

The purpose of the C++ parser—be it the current one, a rewrite, or Clang—is to provide ENIGMA's EDL parser with information about available types, functions, and other constructs. For example, ENIGMA would not know what var a; meant if the C++ parser couldn't understand var's header, and it would not understand what show_message was if it could not read function definitions. The parser also needs to be able to resolve complicated types for further error checking. Hence, we need some mechanism of parsing C++ sources accurately enough for ENIGMA to produce correct C++ code.

I am faced with two options which I have filtered out as the best.

  • Rewrite the C++ parser to support the new ISO and to better support the old.

    This will mean partitioning ENIGMA's compiler into two projects—one to parse EDL, and one to work with C++—and maintaining them both. I would choose to split the two so that any other party interested in parsing C++ could potentially join in, though it has proven in the past to be unlikely for this to happen.

    The benefits of doing this would include minimalistic size and features tailored specifically to ENIGMA and similar projects, meaning—if I code it right—it could potentially work faster than Clang due to a lack of need to lex and check code inside functions. The resulting project would also be less than a megabyte in size (ENIGMA's current hand-rolled C++ parser is 300 KB of source).

  • Drop the existing C parser altogether and outsource to Clang.

    Clang is an LLVM frontend. Certain members of this forum, I'm sure you're aware, blow lots of smoke about LLVM, but in general we would want to avoid it because it will run users somewhere between half a gigabyte and a complete gigabyte of disk space, and MinGW LD/MSys would still be necessary (which is presently the largest component of ENIGMA).

    Clang, and its support for LLVM, would bring unprecedented benefit to the project, but not unwarranted benefit given its size. If ENIGMA was compatible with LLVM, by nature of the huge amount of support for the LLVM project, ENIGMA would be able to use, export to, and interact with a half a dozen other languages, notably JavaScript, Lua, Python, and an interpreter for C++ which, depending on its speed, could mean a native method of doing execute_string(), or could just be something crappy to avoid (we have only just learned of its existence, and I have little faith in the ability of anything to both parse and interpret C++ in a reasonable timeframe). In short, we would be getting not only C++ functions, but Lua and JavaScript functions as well, if desired.

    The issue is not only the huge size. Ostensibly, I could invest a few hours each update into isolating the segments of Clang required to simply parse C++ and give me info about it. Therein lies the issue; I am well aware that at this point, Clang sounds like the clear choice, but whether or not I choose Clang, I am left with something to maintain. Getting just libclang, its Clang dependencies, and the necessary headers from the LLVM svn to compile alongside ENIGMA will take work, research, trial, and error, and will cost about 50MB in the SVN. Then any time Clang updates something, I have to try updating my copy without stomping all over the modifications. I can't measure at the moment how messy it will be; only that the process is not streamlined.

That said, I am torn between the two options. I need people to say, "50 MB and a potential shitton problems that aren't yours, in exchange for four languages? That sounds worth it to me!" Or to say, "for that price, just write your own."

TL;DR version:

(1) Custom(2) Clang
Tiny (Less than 1MB); fast, pointed runtime50 MB; Parses EVERYTHING, though quickly
Gives precisely the needed information, no more, no lessGives general information that can likely be used to meet all of ENIGMA's purposes.
Supports interfacing with other languages (Lua, Python, JavaScript) at the cost of hundreds of megabytes on top of Clang
Likely to be sole maintainer, responsible for all aspects including any potential errors. This would be no different from now. At worst, it could mean a second recode in the future, but ideally I would make the code sufficiently extensible to prevent that this time.Maintenance involves separating Clang from LLVM as cleanly as possible every time an update is made; any parse errors are not the responsibility of the ENIGMA team, and may or may not be dealt with in a timely manner. Potentially, we'd be facing another MinGW fiasco. (See #13297)

Additional Q/A:
dazappa: Clang is a "frontend to LLVM." So you would use Clang but not LLVM? And what's the final size decision, 1gb or 50mb?
Josh@Dreamland: Well, clang has LLVM dependencies, so I would be cutting LLVM into little pieces and throwing away the ones I don't need. 50MB is the projected size after I throw away the little pieces.

dazappa: Would you rather maintain 50mb of shit that you don't know, or 1mb of shit you do?
Josh@Dreamland: Good question. Ideally, for Clang, the maintenance would just be updating Clang headers, adding any more pieces of LLVM that become necessary, and making sure it compiles as though the configure script had run. Maintaining a megabyte of C++ parser can be just as difficult, if not more so as my responsibility extends beyond making sure it simply compiles.

dazappa: Well, you failed to write the C++ parser happily the first time, so you think you'd be able to do it better the second time? Using clang might save you time if you can get it setup and be able to easily update the headers like you think.
Josh@Dreamland: I do think I'd be able to do better the second time. As you can see, the current version succeeds for the most part, with one warning it throws three times. If I code the second version knowing everything I do from the first, and with all that shit in mind, I should be able to get it to play nice.

dazappa: ...Until you realize you have to rewrite it for a 3rd time.
Josh@Dreamland: I'm going to use a system very similar to the recursive descent scheme Rusky talks about. Basically, it would use the body of the current C parser, which invokes a function to handle the token in the context of each statement. Instead of calling this big mash-up switch statement that makes a hundred if checks, the parser would call one function based on the current context and pass it the token, and that function would deal with it appropriately. This means adding a new type of statement to the list would be pretty easy.

Go to town.

General ENIGMA / Re: Makefile is fucked
« on: October 28, 2011, 06:20:45 PM »
So apparently, Rusky's makefile doesn't agree with Mac. I'm not sure what to try; I'd need access to a Mac to try to get them working.

Off-Topic / Re: Compiled Code in GM9
« on: October 25, 2011, 04:40:40 PM »
Heh, this should be good.

Proposals / Re: Overworld Resource
« on: October 19, 2011, 08:37:08 PM »
I was hoping someone else would suggest that. While that was my original idea, I thought it kind of overcontorted the purpose of most RPGs, but I had mostly older ones in mind. (RPGs, and other games with such needs.) For example, I considered LoZ, too. Most of the time, converting coordinates in one room to those in another is all it takes. But I guess even in Zelda, there were stairs that led odd places.

But yes, teleporters would be really nice; the issue is, how do we sync that up with the room system, and how do we put it to functions? At the moment, I'm really tired. My brain hardly works; it's in write-only mode at the moment. But I'm having trouble thinking of a few concise functions to serve those purposes.

I guess, when I think about it, doors like in Kirby's Adventure would be better pulled off with that system than by overlapping rooms in the editor.

Okay, how about this:
In addition to regions, rooms can define entry points and exit points by a given name. The overworld resource then uses those points, allowing users to link them between rooms by drawing arrows between them. We could then offer functions to go to follow the route pointed to by one such node. Or, in the case that there are multiple, offer an overload of that function that uses a different index, for doors and teleporters that go random places.

General ENIGMA / Re: Compiling game without console.
« on: October 19, 2011, 10:14:14 AM »
I was thinking just an integer ID from 0 to number_of_regions. See the proposal I just posted, and argue for name.

Proposals / Overworld Resource
« on: October 19, 2011, 10:13:28 AM »
Formal proposal time, now that EGM is finished.

Anyone who has tried to make large, complicated rooms in Game Maker has been either bitterly disappointed or ingeniously crafty with regard to GM's rather limited room editor.

The quintessential example of a game that's fucking hard to make in GM is a Metroid platformer. There are two ways to go about the layout of a Metroid game in GM: There is the traditional method of creating a room for each little passage and cranny and using a complicated network of global variables to determine where you are coming from and where you are going, or there's the perhaps more advanced method of creating a large room of room-editor-lagging proportions and then using instances to denote regions to activate or deactivate as needed.

I have, in my longish and complicatedish game making history, made both a 2D and 3D Metroid Platformer, using a version (albeit a crude one) of the above strategies. It's quite possible that I've missed a strategy more form-fitting of Game Maker's... capabilities, but even in retrospect, I am not seeing a better way of pulling it off.

Enter EGM. Enter Overworld Resource.

Now that the EGM format is complete, we are capable of extending the existing room resource and adding a new resource on top of it.

Extending the room resource, we would allow for users to define their own "regions," as MrGriggs called them when he suggested them. These regions would be simple geometric shapes (possibly just box and circle) which, using a new set of room_region_* functions, the user would be able to test for being inside of. We then could have an instance_(de)activate_region function. Other modifications could be made to associate certain resource groups with regions, but this is part of another proposal I should probably post next.

Now, for the new resource, Overworld. Essentially, the overworld resource would allow users to place rooms into a larger room editor, the overworld editor. Ism would pre-render low-resolution previews of the room (she could fetch larger previews upon zoom) and allow users to place them in correct proportion on the Overworld map. A set of functions would be provided to get the full rectangle coordinates of a given room in the map, and contiguous rooms could be mapped together to allow quick searching for neighbors.

With that in mind, rooms could be easily reused without a complicated network of user-defined globals, such as room_to_left, room_to_right, room_above, room_below, or, worst of all, x_coming_from, y_coming_from.

Using the Overworld function set, a user could convert x-y room coordinates to x-y overworld coordinates by adding overworld_get_room_x(room_in_overworld) and overworld_get_room_y(room_in_overworld). The user could then translate those coordinates in the appropriate direction so as to leave the current room (essentially walking the character over), then make two calls to overworld_ functions to determine which room is actually going to be entered, and convert the overworld x-y pair back to room x-y pair after the room has been reassigned.

This topic is a good place for argument as to the exact functions to be made available, and this wiki page is a good place to put the results: http://enigma-dev.org/docs/Wiki/Overworld


Issues Help Desk / Re: svnkit error
« on: October 19, 2011, 09:43:23 AM »
MinGW-get is having problems at the moment, because the MinGW team is understaffed or incapable (I know the former is true; I scream about the latter sometimes when bugs surface).

Other than that, your problem has all the symptoms of someone checking out from the SVN and calling Java on LGM manually. On most platforms, that's an option, but on Windows, we have to assume that the userbase doesn't know how to do that and so distribute install zips.

Edit: Let me clarify. I said "Other than that" because if the MinGW team had fixed their issues in the meantime, without my knowledge, that is the assumption I would have made. However, it seems mingw-get is still very much broke, and so you will either have to run their manual installer (selecting msys and gcc) or just wait for them to fix that. They're not a very responsible team.

General ENIGMA / Re: Compiling game without console.
« on: October 19, 2011, 09:13:32 AM »
ENIGMA uses new for each instance, rather than new[] on several. Normally, you are right; requesting a contiguous plot of memory at once instead of asking 10 times for plots 10% of the size is much faster, but you will only see that overhead, ideally, once per instance on the screen. Destroying an instance and then creating a new one of the same size has less overhead, if I recall correctly.

However, allocation times don't really worry me. If they did, I could come up with a far better way of handling this on systems with large memory and slow allocation (systems with small memory presumably have less applications to serve it to, ergo would be quicker on the draw, I would think).

Basically, what I would do is allocate our own heap up front and keep a journal of available spots. We would then write our own new using new() (by your notation, I guess this would be new()()). Basically, we would take the max size between all objects, then pick a random heap size (Probably of 20-40, but I think it'd be wise to give users a say in this), and allocate a grid that way. The journal would then be populated with available spots on the grid. Allocation would be very fast, and deletion would simply queue up a new slot on the journal. As you can tell, the journal would probably be a queue or stack.

As for room regions, I think I know what you're talking about. In a Metroid game I made once, I needed to deactivate all instances except for those contained in rectangular sections of the room. What I did was create two objects with different sprites, one representing the upper-left corner and the other representing the lower-right. I would place these instances in the room and then use their room creation code to group them. Telling which regions you were in would then be as simple as iterating them and testing a few x > other.x. However, ENIGMA presently doesn't support instance creation codes or instance deactivation, so the point is presently moot.
Anyway, is that what you are talking about by regions? The room editor is about to undergo some rethinking as we add the new ENIGMA resource, "Overworld." Feel free to make suggestions on that. I will post a topic with regard to the new resource on proposals; I invite you to either respond there or to create a new proposal topic on the matter.

General ENIGMA / Re: Compiling game without console.
« on: October 17, 2011, 01:19:56 PM »
I thought it was January.

General ENIGMA / Re: Compiling game without console.
« on: October 17, 2011, 11:38:33 AM »
We're working on it. Should be done by the end of the week.

Tips, Tutorials, Examples / Re: How to properly use OpenGL
« on: October 05, 2011, 10:47:01 AM »
I don't know if you've noticed, but the macro already does that.
#define untexture() if(enigma::bound_texture) glBindTexture(GL_TEXTURE_2D,enigma::bound_texture=0);

It's done that since the dawn of time.

While it's possible that VBO, being newer, could be faster than lists, I'm mostly concerned about support. Deprecated, maybe, but still supported to at least the point of working on the shittiest cards with the shittiest drivers. Any card that gives GL lists no boost probably doesn't bother offering the VBO extension. On top of that, I have no benchmark to suggest superior speed either way. Perhaps once I've actually implemented tiles, I can move to Windows and test one list implementation and one VBO implementation. I've got a pretty new graphics card (few months old), so I'm sure it'll be as fair a representative as one card could ever be.

Tips, Tutorials, Examples / Re: How to properly use OpenGL
« on: October 04, 2011, 07:15:07 PM »
I was talking to HaRRi, luis. :P
He was the one that mentioned my name.

It may prove that VBO is more widely available as an extension now that Microsoft has their head out of their ass GL-wise. In the meantime, the deprecated, proven method is the way to go.

I don't believe for a minute that you've found any situation in which VBO would accelerate primitive drawings and lists wouldn't.

Tips, Tutorials, Examples / Re: How to properly use OpenGL
« on: October 04, 2011, 05:54:40 PM »
I told you that I intend to use lists for tiles. I'll bet you a hand of bananas it's faster than a VBO.

If you find a way to remove the texture flipping at the beginning of each draw function, you let me know.

Proposals / Re: HTML5 exporter
« on: September 27, 2011, 04:41:42 PM »
Nah, JavaScript already has plenty of wrappers to the C++ map structures.

Proposals / Re: HTML5 exporter
« on: September 27, 2011, 04:02:47 PM »
And the JVM is made of non-C++ metaphysical goo, right?