Prince of all Goldfish
Location: Pittsburgh, PA, USA
Joined: Feb 2008
I like the idea, and encourage you to go for it. Macros in C++ can do a lot of what the parser does, if you keep your code syntactically correct. It's a bit ugly to do that, but that's what it takes, sometimes.
Please don't get me wrong—I also see ENIGMA as a diamond in the rough. But I believe a huge contributing factor to that was the IDE. However, even implementing a GM-like engine for single applications in other languages led to simpler, more beautiful code for me, in the past, so there is probably something to the idea of trying for a pure-code implementation.
I'm sorry if I give the impression of not caring about the project. While you have been looking at the engine and seeing the potential for what it could be without Game Maker holding it back, I have been looking at the frontend and imagining what it could be. I think you'll find that in both cases, offering about 85% feature parity is trivial, and as you hinted, that's all ENIGMA has really ever done.
So, by all means, keep us posted. Know that on the IRC, ideas are constantly being kicked around for what an IDE should look like. Robert's off to college, and other maintainers are getting on with their lives. It might surprise you to learn that Quadduc is actually hanging around the IRC, though he doesn't say much. Sad thing is, I think others (particularly Robert) are waiting for me to make a move, and I'm waiting for confidence that I'm not going to blow it. As my life at work continues to solidify into something that I don't have the power, and therefore the need, to influence, I am beginning to invest more and more time into thinking this through.
I'm going to go ahead on a bit of a rant, because I find it therapeutic. You can probably stop reading here.
I recognize that I am making a classic mistake of overthinking actions before taking them. As a kid, I didn't have this problem. Not to take credit for the countless hours of hard work you've put into this project in the last eight years, but my childlike belief that just slapping code down would make everything work is a huge part of why this project exists right now. And as I work close to other engineers who don't commit anything without PRDs and design docs and tests and all this other administrative minutia that I'd have gawked at as a child, and as Rusky occasionally bombards newcomers with articles from others like us who have discovered how easy it is to point out design practices that are not correct, I continue to think about the ones that are. I guess at this stage in my life, ENIGMA as it stands today is the answer to "What would happen if I just cowboy-coded everything?" ...and I remain unconvinced that it can ever be the best answer. Or even close to it. I'm not looking for mathematical perfection; I'm looking to remove all these little gotchas and bugs.
When I was younger, I always felt like this was all fine, because the things that suck can just be recoded. But I recoded that damn compiler, what, four times since I was 15? Each time I came into it knowing five times as much as I did the last time, easily. Hell, the first time I wrote a GML→C++ parser, it was in GML. I knew hardly any C++ at the time; I was fourteen. Talk about bootstrapping.
But now that I'm a full-time employee, working for a company that maintains projects dozens of times larger than ENIGMA, and am surrounded by people who all share the same fears that started forming in my mind when I was 20, I'm seeing the costs that came about every time I recoded the parser. Each new version did a dozen things better, and exhibited a half a dozen regressions. Did you know that the second time I recoded it, I forgot to add comments until I tested it out on a full-blown game?
You might be thinking, why not just recode pieces of it? In general, that's a better idea, but all of these pieces have a fundamental dependency in them that is not easily upheld: JDI. We'll get back to that in a bit.
After all, I'm smarter now; I feel as if I'm a hundred times better at coding than I ever was when I was working on ENIGMA. As a consequence, the compiler seems daunting to me—not because it's so much code, or a hard problem, but because it does so many weird, little things to get everything right, and then it still doesn't get everything right. It never has; it's always been some weird maximization problem that I never fully established had a solution. Because as a kid, I didn't plan the whole project out; I just built the pieces of it, and they indeed fit together into something that looked like what I set out to build. It wasn't just "build a compiled Game Maker." It never was. That's an easy project; it'd take me a few weeks to do that. It was "build something that looks like Game Maker, but compiles to C++ and is really fast." I could even do that much now, with a few executive decisions that would just be made. But then, look at "really fast." What does that even mean?
I know you were able to get your tool to run really fast in ENIGMA, but ENIGMA is a game engine. Part of making ENIGMA really fast is handling things like sprite batching (which Robert implemented something like) and spacial partitioning for collisions (an idea that as a kid, I could never work out). Even now, I don't have a great answer for how to do it seamlessly—without the user having to manage it—in a way that's actually efficient. Does GM even do that? For fuck's sake, Dr. Overmars was teaching a class on collision handling. That was his thing! Yet he designed GM to use an x variable and a y variable. Now I have to update the quad tree twice every time you move the object yourself! It's that kind of little shit that bugs me about this whole operation. How do I design around that? Do I choose to just not care? Kid me wouldn't have thought about it for more than ten minutes, I suspect, as much as kid me loved efficiency. But then, kid me wasn't so great at that, anyway.
I made a lot of weird decisions as a kid. Did you know the first local variable access code literally copied all the variables out into temporary storage so they could be accessed generically? The entire instance was copied somewhere every time you needed to access some_obj.x. This only existed for a few weeks, before I coded the current implementation, but even now, I'm not sure that the current implementation is the best way. Look how fast some of these new scripts are, using lookup tables for their variables. That would have made things easier.
Not to mention the decisions I made involved maintaining a C++ parser for ENIGMA's purposes. If anything is going to show you how dangerous an idea that is, it should be C++11, which JDI still doesn't support. As much as I think LibTooling is a fucking cow, it would have been worth using just to narrow the scope of the project. These words are bitter and they hurt me as I type them; it's such a vapid, meaningless statement, until you realize just how much work actually goes into maintaining all these little-shit dependencies. You just have to accept you can't have a non-bloaty ENIGMA compiler while still supporting the ever-changing C++ specification in the background.
It's 2016, now, and LibTooling still isn't where I needed it to be as a kid. My argument for keeping my C++ compiler (JDI) around was that it not only granted us the definitions we needed from the engine, but a lot of the logic needed in parsing C++ was also intrinsic to EDL. Consider overload resolution: ENIGMA has really poor overload handling, and for a while, I expected JDI to be the cure for that. It was unfortunate coincidence that I finished JDI just as C++11 was coming out; if I had known all the new features of C++11, and baked them into JDI, we might be having a very different conversation right now. But then, if I'd known those features, maybe that'd leave me where I am today, and I wouldn't have written JDI at all, for an understanding that maintaining two compilers just isn't a good idea—even if they're compiling virtually the same language.
Perhaps instead, I'd have modified LibClang to add support for with(). I'm not sure; I was an industrious kid. It still wouldn't have bought me out of the quad tree problem, but it would have at least spared you the problems you've been having.
The one philosophy I had as a child that I will always keep with me is that when you design something, you need to take a step back, stop thinking in terms of the tools you have, and start thinking about the best possible way to say something. Thinking outside the box, as it were. I remember designing ENIGMA's show_menu functions in GTK. I'm still a little proud of them, even though they're a little backward. It doesn't meet every use case, but it meets a few of them superbly. And even now, at this huge company I work for, that's exactly what the engineers around me seem to be trying to realize.
So I'm looking at the big picture—bigger than I ever looked as a child—to find what I really believe is the best way, end to end, to make a damn game. And if I find it, you can bet your ass that I'll be back.
That's enough rant. The point is, I don't have all the answers, anymore. In truth, I never did; I was just happy to keep working without a plan, and that's changed for various reasons, but possibly not right reasons. Some soul searching is still in order, I suspect. I can't make you any commitments, other than that I'll be around.