ENIGMA Forums

General fluff => Announcements => Topic started by: Goombert on May 03, 2018, 08:36:30 pm

Title: RadialGM
Post by: Goombert on May 03, 2018, 08:36:30 pm
(https://enigma-dev.org/docs/wiki/images/6/6b/Rgm_ide.png)

Ok, so I've been hinting around and letting some of you know for a while that a few of us have been redesigning the architecture of ENIGMA to support a new IDE. Some of you may be more familiar with it as I know I've let a handful of people test out the new tools, but I'm here to break it down and try to explain it in more detail for everybody.

History

I want to start off by enumerating the current things ENIGMA was doing that are somewhat insane. When I started contributing to LateralGM, GM HTML5 was a brand new thing and there was really only the GMK format to support. This went well for a time, as many of you know, and we were able to add features as YoYoGames did without too much of a thought-out serialization framework in place. We had experimented with making a new IDE, just for the sake of making a new IDE (not as much because one was needed). Working on LateralGM, I tended to continue with the mindset that it was a "GameMaker" IDE and we should try to add compatibility for everything which is now not only a difficult task but next to impossible. Because LateralGM and ENIGMA communicated using C structs with JNA through compileEGMf.dll, they would not be binary compatible if we added new fields and other metadata. Overtime, Oracle has become an even worse company attempting to undo decades of legal precedent (https://mashable.com/2018/03/30/google-vs-oracle-explainer/#EQFVoM02wPqH) with regards to software engineering, Java, and specifically OpenJDK, hasn't made fast enough progress, and I just want to make games.

Architecture Changes

So I experimented with using Google Protocol Buffers (https://developers.google.com/protocol-buffers/) as a serialization for projects. I was also adding things here and there to emake.exe (https://enigma-dev.org/docs/Wiki/Command_line_interface) as fundies asked me to. He was working with Josh to get our CI tests setup to prevent regressions in the engine and everything (which I must say are doing wonders for this project). fundies expanded the protocol format to basically load GMXs completely and then I tacked on GMK, have YYP semi-working, and him and Josh are currently working on EGM saving. These formats are all being handled in libEGM which is a new C++ library built on top of libProtocols.

The plan is to have libEGM only ever read the GM formats but save and read our official serialization format (*.egm). The CI tests will make use of libEGM to load and compile a test project from every version of GM back to GM5. You can actually examine this right now on GitHub because the Protocol Buffers PR is already running these tests:
https://github.com/enigma-dev/enigma-dev/tree/gmxProtoButt/CommandLine/testing/SimpleTests

RadialGM

Using the new protocols I was able to experiment with making RadialGM actually functional (since because of the architectural changes much of the work was already done).
https://enigma-dev.org/docs/Wiki/RadialGM

As you can see, we've made a lot of progress. I am really satisfied with the results so far, and personally, I want to make games myself using the new IDE and that's why I've been working on it so much. What's so great about all of this though, is not just that RadialGM is actually possible, but the architectural changes really do make every other IDE as equally likely to work out (though for various reasons, I've decided to focus my efforts on the Qt version; simply because I personally like it the most). However, you should consider RadialGM to be more of an "ENIGMA" IDE as it will be almost purely for ENIGMA editing.

What You Can Expect

* It is unlikely that I will be making any more changes to LateralGM.
* The C-based backend/ in ENIGMA will remain but may be rewritten to convert to the protos (instead of the inverse) for the foreseeable future. It will likely be deleted if someone decides to rewrite the ENIGMA-LGM plugin to use the protos instead of the C backend (but that's much more work than converting the backend into protos).
* You can expect to be able to load all GM formats back to GM5 (*.gmd, *.gm6, *.gmk, *.gm81, *.gmx, *.yyp, *.egm).
* Do not expect to be able to export any RadialGM project back to a GM format, you will likely convert your projects to EGM upon open/load and then either maintain two copies or stop using GM.
* You can expect the IDE to make more optimal use of your system's resources for the above reasons (primarily because resource data is lazy-loaded and reference counted in the frontends).
Title: Re: RadialGM
Post by: egofree on May 04, 2018, 04:58:42 am
Hi Robert,

Thanks a lot for the update and your commitment for the project. Would it be possible to have a link for an executable of RadialGM ? I don't think everybody wants to compile the project.
Title: Re: RadialGM
Post by: Goombert on May 04, 2018, 06:00:52 am
Hey egofree! I'm glad you caught wind, how have you been?

Back on topic, and in regards to your question:
I want to clarify to everyone right now, this announcement is purely to get everyone informed. I'm not actually looking for testers yet (but privately I will give the directions if requested) - there will be more announcements to come (and very soon too!).

There's just so many technicalities we felt it best not to spring it all on everyone at once.
Title: Re: RadialGM
Post by: egofree on May 04, 2018, 10:27:47 am
Hey egofree! I'm glad you caught wind, how have you been?

Back on topic, and in regards to your question:
I want to clarify to everyone right now, this announcement is purely to get everyone informed. I'm not actually looking for testers yet (but privately I will give the directions if requested) - there will be more announcements to come (and very soon too!).

There's just so many technicalities we felt it best not to spring it all on everyone at once.

Ok, i understand. Thanks. Last year i worked in my free time with Unity to create another remake, but now i've less time. I had a lot of fun doing it. Anyway Enigma will always stay dear to my heart, so from time to time i visit enigma's website.
Title: Re: RadialGM
Post by: time-killer-games on May 04, 2018, 10:38:59 am
You still didn't answer his question though, Goombert. When it is released, will there be a standalone available for each release, or will everyone be forced to compile it from the source code themselves? :P
Title: Re: RadialGM
Post by: Goombert on May 04, 2018, 04:28:05 pm
Quote from: egofree
Last year i worked in my free time with Unity to create another remake, but now i've less time.
Do you mean like a game remake or another thing or like how ENIGMA is a remake of GM (really it's an augmentation... not a remake)?

Quote from: egofree
Anyway Enigma will always stay dear to my heart, so from time to time i visit enigma's website.
Thanks dude, you were always really nice and helpful here as a contributor. Please do come back too because I do foresee this thing becoming stable and I hope you'll be around for the beta. I honestly can't predict much right now, but I've been pretty consistent about sticking to an end-of-summer deadline to get a beta out.

Quote from: time-killer-games
When it is released, will there be a standalone available for each release, or will everyone be forced to compile it from the source code themselves?
I'm sorry, thanks for clarifying. I can't say just yet, but yes I would like to see a real installer for ENIGMA, but we have to look into how to make that possible with msys2. The setup will certainly be a lot easier. I have to look more into the technical details and also wait and see because there are other things I want to do. But yes, definitely would like to see easier installation.
Title: Re: RadialGM
Post by: egofree on May 05, 2018, 03:50:14 pm
Do you mean like a game remake or another thing or like how ENIGMA is a remake of GM (really it's an augmentation... not a remake)?

I started to develop a remake of the c64 game Wanted : monty mole : You can see how the original game looks on a emulator here :https://www.youtube.com/watch?v=r26norwZ_XI (https://www.youtube.com/watch?v=r26norwZ_XI)

I consider that i've done about 40% of the project. Even if it's a very old game, i had a lot of fun to try to understand how the game was made. For instance, every time you start a new game, a new maze is generated in the middle of the map. I spend several weeks to understand how the maze was generated. I wrote a small article about the maze generation there : http://www.lemon64.com/forum/viewtopic.php?t=64877 (http://www.lemon64.com/forum/viewtopic.php?t=64877)

From my point of view, game programming is really the most interesting and fun aspect in programming. You can find exciting topics like AI or 3D graphics. Soon i will start again to develop this remake in my free time. What is dangerous is that when you start to program games, it's very hard to get a life and relax. It's like you have always a thread running in the background of your head which is always thinking about how to solve problems and bugs. :D So even in the middle of the night, it's very hard to sleep, as you have new ideas coming out, and you want to test them as soon as possible !   ;D
By the way it's totally offtopic. Sorry  :ohdear:
Title: Re: RadialGM
Post by: Goombert on May 06, 2018, 12:28:22 am
I had not actually heard of that game before, but you can't blame me too much because C64 wasn't my generation, N64 was :P

Regardless, that's really interesting. I don't think I've studied much into maze generation, but one thing I was always interested in was world generation (think Dwarf Fortress). I was always into doing that for both 2D and 3D because it always seemed less linear to me, and I don't know, I've always had a bias against linear games (but not against 2D).

And yes, I happen to agree with you as well, video games are always the cutting edge of computer science, there's always something interesting going on. There's AI, 3D, 2D, art, modeling, sound recording, music synthesis, plain old programming, networking, data structures, and everything else. I must say I've experienced what you describe about a thread with writing this new IDE on several occasions. Sometimes you get stuck on how you want to do something and after thinking about it for a while, it will "just click" and you suddenly know exactly how you want to do it.

But no, don't worry about being a little offtopic, a lot of what you are saying is relevant to where we're going with ENIGMA and a new IDE too.
Title: Re: RadialGM
Post by: time-killer-games on May 06, 2018, 06:47:43 am
Only fair to warn you - no one new will use ENIGMA, even with RadialGM, if you force them to compile from source.

Ain't nobody got time for that!
https://www.youtube.com/watch?v=zGxwbhkDjZM
Title: Re: RadialGM
Post by: Goombert on May 06, 2018, 08:34:56 am
As I've been saying from the beginning, it needs investigated more, but it's definitely something I want to look into, a native binary installer. You just can't make an installer before you actually make the IDE, hence, we'll get to it when it becomes important. The only people actually needing to set it up right now are those working on it, and for that, the MSYS2 installation works very well and much better than our old solutions (due to the package management).
Title: Re: RadialGM
Post by: BPzeBanshee on May 10, 2018, 12:31:08 am
Been out of the loop for a little while until I received an email notification about an old GitHub issue I reported last year. This work on not only a new IDE but a consistent and stable way of using ENIGMA sounds promising and if I don't have to jump through a stupid amount of hoops to get it to work like I did last time I'll be happy to test my projects on it again in the future. :)

Quote
* Do not expect to be able to export any RadialGM project back to a GM format, you will likely convert your projects to EGM upon open/load and then either maintain two copies or stop using GM.
This has me somewhat concerned though. I think I understand the reasons for why writing support is being dropped given YoYo's tendency to change how they themselves write to it, but EGM has never been reliable in the time that I've spent looking at this project and as a binary flat ala GMK I'm not sure limiting export options to that is a good foundation. Would it be out of the question to at least consider GMX (Studio 1.4) write support as an alternative to it given that GMS 1.4's update support ends this month and therefore won't be subject to the above issue with YoYo?

Also, will RadialGM work like LateralGM in that it'll function for at least the purpose of opening and viewing projects without ENIGMA necessarily set up? This and the GMK format support were LateralGM's biggest strengths.

As for the subject of installers and the like, I'd be more than happy with a portable ZIP solution as long as it works. The existing portable ZIP from last year reminded me somewhat of MAME's environment system (http://www.mamedev.org/tools/) but where MAME had clear instructions and had the needed components there to go, ENIGMA required research through multiple pages and extra components to slot in from Dropbox links.
Title: Re: RadialGM
Post by: Goombert on May 12, 2018, 06:10:01 am
Hey BPze! The issue I closed I hope I did so in good faith because I will get around to those missing events, just trying to organize all the open issues. A lot of issues being closed have actually been resolved, some by GCC themselves, some by us, and some didn't have enough info to go on.

Quote from: BPzeBanshee
This work on not only a new IDE but a consistent and stable way of using ENIGMA sounds promising
I really hope so, what I've been trying to express to people is that this time will be different not just because we're all more experienced programmers. We have this down to a science now thanks to continuous integration where we can verifiably fix bugs and create a test that, like I've been saying, makes sure they never come back.

Quote from: BPzeBanshee
stupid amount of hoops to get it to work like I did last time I'll be happy to test my projects on it again in the future.
I agree, and we're still thinking about this problem, but the new setup I am using is MSYS2 based. So, what we want to shoot for will probably be something a lot like MAME. I am actually glad you linked that because I've actually already looked at it for this haha. But most importantly Java won't be required unless you want LGM over RGM, that's clear already.

Quote from: BPzeBanshee
I think I understand the reasons for why writing support is being dropped given YoYo's tendency to change how they themselves write to it,
I know that this is a hard one for people to grasp at first, but it doesn't mean it won't be possible. It just happens that it won't be easy to export back to GMX because we have, for example, no need to use proto options to distinguish which fields are nodes and which ones are attributes. In order to write GMXs, we would actually need that kind of field description. In other words, the writers are much uglier than the readers (but I will tell people how to create/code them if they want to do so). However, I may not allow them or protest if somebody tries putting them into the ENIGMA main repo.

This hits right at the central idea of RadialGM, the fact that it's an "ENIGMA" IDE and not a "GameMaker" one. Doing this does simplify a lot, and not just the readers, but also the complexity of the UI. For LateralGM to do what it did, it had to have a UI option for everything every GM format ever supported (and that kind of makes its UI ugly).

Quote from: BPzeBanshee
but EGM has never been reliable in the time that I've spent looking at this project and as a binary flat ala GMK I'm not sure limiting export options to that is a good foundation.
I can understand these reservations due to my own mishandling of the lgmplugin in the past. But the idea is to make the format totally complete from the very beginning and only rarely needing changed. When it does happen to need changed, the protos make that easier because they are automatically backwards compatible and binary compatible with ENIGMA itself (hence no need to update a plugin or jar file). Also, we will ofc have an EGM project in the CI testing this time around to make sure silly stuff doesn't happen. Finally, it's not limiting the export actions as I described, just limiting the save function (I am conceptually making a distinction between exporting and saving here).

Quote from: BPzeBanshee
Also, will RadialGM work like LateralGM in that it'll function for at least the purpose of opening and viewing projects without ENIGMA necessarily set up? This and the GMK format support were LateralGM's biggest strengths.
This one is a bit of a tough question I don't necessarily want to answer just yet. I want to say it's unlikely because the protos themselves as well as libEGM (which loads GMK etc) resides in ENIGMA's main repo. The reason is because emake itself depends on it and we use emake to test ENIGMA from Travis/AppVeyor to prevent it from breaking. So the protos again, pretty much "belong" to ENIGMA and ENIGMA now actually understands what a GM project is. Hold that thought for a second, when I consider this myself, in retrospect it seems really insane that previously ENIGMA didn't understand any GM projects when it itself is a GM augmentation.

Quote from: BPzeBanshee
extra components to slot in from Dropbox links.
Yeah, there won't be any dropbox links unless you go the LGM route. And we'll probably submit an actual ENIGMA/RadialGM combo package to the MSYS2 maintainers.

I hope that answers most of your questions and gives you a little more insight into the thinking behind this/the justification for these decisions. For the most part, all of this will be worked out in time, and I definitely hope you get a chance to test this thing (Y)
Title: Re: RadialGM
Post by: Steve on June 17, 2018, 04:45:08 pm
This looks great!

Out of interest, what did you use to create the IDE?
Title: Re: RadialGM
Post by: Goombert on June 18, 2018, 01:31:12 am
Qt Framework is being used for the GUI controls and it's using a data model layer (from MVC architecture) that is powered by Google Protocol Buffers.
Title: Re: RadialGM
Post by: Steve on June 18, 2018, 09:43:30 pm
Truly impressive effort!

So much cleaner than what GMS1 ever was and much nicer than the debarcle that is the IDE of GMS2.
Title: Re: RadialGM
Post by: pavul on June 22, 2018, 03:58:12 pm
i think you are doing a great job, however i dont know if will be open source as well and i would like to see the improvements, if is an IDE for enigma with the UI to create a game like GMS but without unusable items would be better, for that i dont use D&D i dont need those bunch of icons, only script and the events.

and im from now on waiting some advancements.
Title: Re: RadialGM
Post by: faissaloo on June 29, 2018, 03:25:21 am
i think you are doing a great job, however i dont know if will be open source as well and i would like to see the improvements, if is an IDE for enigma with the UI to create a game like GMS but without unusable items would be better, for that i dont use D&D i dont need those bunch of icons, only script and the events.

and im from now on waiting some advancements.
The source is here: https://github.com/enigma-dev/RadialGM/
Title: Re: RadialGM
Post by: Goombert on June 29, 2018, 08:57:30 am
Quote from: Steve
Truly impressive effort!
Thanks Steve! We're really trying, and it wasn't always this great for ENIGMA either. What's really helping is our embrace of continuous integration and regression testing. The changes are truly revolutionary for this project. Please check out my other announcement for more on that.
https://enigma-dev.org/forums/index.php?topic=2904.0

Quote from: pavul
however i dont know if will be open source as well and i would like to see the improvements
Fear not, as we all intend to retain the open source nature of RadialGM and ENIGMA as well. Open source has always been a defining principle of this project and none of us have any intentions of changing that.

Quote from: pavul
if is an IDE for enigma with the UI to create a game like GMS but without unusable items would be better, for that i dont use D&D i dont need those bunch of icons, only script and the events.
The initial beta is going to lack a Drag & Drop UI completely for the time being. Games like this will continue to function partly thanks to the fact the Drag & Drop in GMS2 is actually embedded as regular GML and using a document generator to read metadata out of the comments. That allows it to completely function in ENIGMA without any UI when I get the YYP reading finished. I will continue to look towards keeping a non-drag and drop mode for users who prefer that if and when a drag and drop UI is developed here.

Quote from: pavul
and im from now on waiting some advancements.
Hopefully much more on this soon, we're almost done with the engine changes. Please read more about that on the announcement I linked for Steve above.
https://enigma-dev.org/forums/index.php?topic=2904.0