LateralGM seems a little clunky overall, Ideas but not sure how to develop them.

[HitCoder]

I have ideas for a new IDE for Enigma, LateralGM just doesn't do it for me; it looks like Game Maker, don't get me wrong, and it's better than GM itself when running in Linux, but in windows it's horrible, it's clunky and it's unstable. I don't have any logical theories for these differences however; I still think it would be nice to have a new IDE, and I have plans for how it should look in my head, which I will spend time drawing and annotating for you all, but graphic editing software will make this a gruelling process, so I may make a concept design within Enigma, which if I can design it exactly how I want it designed, if some people would be nice enough to hook the functions up to the system of Enigma and all the compilers, I'd be really happy. I love Enigma, and it's community, and it's developers, maintainers and contributors. The whole project is amazing, and it makes me sad how nobody notices the whole thing, there is so little attention to this project, especially nowadays since people's lives are getting in the way of development.

I really think LateralGM has so many issues (primarily on windows) that are just a pain to get fixed, I have seen The ExDeus's future aspirations for the project and I personally really like them, I would like to see Enigma become much more independent; it does not need to continue with GM compatibility, it can be so much more, so much more potential is here.

Now, I understand people in this world tend not to have much time to do things in life, especially how most people in the community and contribution circle of the development of the project are getting older and getting more involved in life things, it's more and more difficult for you to maintain the project.
I have considered learning a bit more on Java to help develop LateralGM, but I really dislike Java's GUI handling; I hate the whole thing with the clunky theme setting, etc. I think something else should be used to make an IDE, not Java; Java has problems.
I read somewhere that one of the developers/contributors would like to see an IDE developed in Enigma; this implies to me that it would be possible, and if it's possible, I would love to get involved.

Now, here is the way I work with code, just in case people need this info to know whether or not they want to get involved in my idea for this project; I will probably end up making the GUI, and get stuck on certain things.

Good things:
 - I am very methodical, I have to be very careful in how I code things.
 - I have good concentration, I will work and work until something is finished. I have even forgotten to sleep whilst working because I feel that I need to finish
   things.
 - I can comment code that is presented neatly quite quickly, this does not need to be when I'm coding, it can be from code that hasn't been commented.

Bad things:
 - I can be lazy, but this is due to lack of motivation; I'm not good as a leader because I can never motivate myself, I lose interest if nobody else seems
   interested
 - Because I have to work until something is finished, often if I'm working on a function or similar, if I leave it and come back to it I forget where I left off,
   meaning that I panic and try to avoid having to finish it. Normally however, I can get on okay if I just remove the code that is unfinished and start over.

I don't know whether this idea is going to be condemned or praised, whether people are going to laugh at me, or even if people are going to dislike me because I'm coming up with a new idea when LateralGM is still being maintained; it's an idea, I'm not saying LateralGM is terrible, I can work with it perfectly fine on Linux, but here are the things I dislike about LateralGM just briefly:

The sprite editor/viewer interface is very nicely presented, much more informative than GM's. However, it's a little more overwhelming, but lacks being able to apply animation effects, which matters not to most, but occasionally I miss these features, especially having to apply the effects manually through the image editor I am using on whatever system (On Linux I use GIMP, but dislike it because it's just an overwhelming interface that I haven't yet got used to)

The code/script editor just has a strange appearence, the font looks too pixelated, the colour coding seems off, and I find it annoying having to press ctrl+space to get suggestions for functions - but I find having it appear automatically annoying within Notepad++, so maybe make it appear automatically after a small pause, and only on unfinished functions, but obviously I'm sure a few people find it fairly annoying, I can see why too.

You may be thinking "ugh, this kid is 15, he doesn't know what he's talking about.", which I understand why you would think this, sometimes I come across completely opposite to how I want to, and sometimes things I say don't make sense - at all.

Obviously this has an included poll, so vote whatever you think - I've never submitted or opened a poll before, I'm assuming you can post replies here, but my bad if you can't x_x

Thanks again everyone, it's great being here.

~HitCoder

[Goombert]

Hey HitCoder, I'll give you some feedback that hopefully gives you a deeper understanding of where we are and maybe even give you some ideas.

For the most part you are absolutely right and we kind of all agree. LateralGM was meant as an intuitive GM IDE. Because it does so well and it's something that people have become used to, I've tried to avoid completely radical changes and instead encourage forks of the project. What bothers me and a lot of people is that the IDE is extremely monolithic. You can't replace a single editor with a different one or add new project formats, the plugin system for LGM is not very well designed. Don't take this the wrong way, but I actually do not agree with the complaints about LateralGM on Windows because of this. Sure LGM may not be designed well for a good plugin, but its problems almost always crop up when used with the ENIGMA plugin on Windows. LateralGM by itself rarely if ever (I've never seen it) segfault on its own all by itself, it's always the ENIGMA plugin crashing it. And this is not just because of LGM's poor plugin architecture it's the generally bad design of the ENIGMA plugin itself.

There's several options to building an IDE:
* Java with Swing (what we have now with LGM)
* Java with JavaFX allowing CSS styling and customization (would be a lot better than LGM; look nicer and modern and less inconsistency in the Look & Feel; would also be faster as JFX is hardware accelerated)
* C# with .NET/Windows Forms or GTK# (might be a little better; C# is not quite as bad as Java and the look and feel is at least consistent with Windows Forms/GTK but there's not possibility of styling (I think?) but you could also just use a different GUI framework but I am not sure what possibilities C# has)
I attempted this with SharpGM and did a number of GM's editors.
http://enigma-dev.org/docs/Wiki/SharpGM

* C++ with Qt Framework (would be fast; would look really nice has less bugs and sometimes looks more native than wxWidgets)
http://enigma-dev.org/docs/Wiki/RadialGM

* C++ with wxWidgets (would be fast, wouldn't be bad at all)
I've also done this one already when I started and was building wxENIGMA
http://enigma-dev.org/docs/Wiki/wxENIGMA

* C++/EDL/GML by building an IDE in GM/ENIGMA (may be kind of clunky and require a lot of extensions; then again there's a lot of the features you need already there; can't really do any super advanced things you may want to do though like radical changes to the architecture; ie would be better done in ENIGMA than GM or by just making ENIGMA its own project different from GM with an IDE in itself like Unity3D)

We've also discussed a native C++ Command Line Interface for improving LGM. This would almost certainly get rid of all of the segfaults in LGM but would possibly be slower in some cases because of the increased disk input/output operations to save the files before passing them to the CLI. Because of this it is better to think of the CLI solution as an alternative but ENIGMA should still support in memory compiling through the compileEGMf library interface. When the IDE has the resources in memory it's actually just quicker to pass them in memory than write and read them back from the disk. The CLI only makes it potentially quicker for LGM because it's Java and passing resources in memory between Java and C++ with JNA is probably really slow because it has to do so much work (probably why big games take so long time to compile compared to reloading them in LGM). I hope that makes sense? It's the combination of Java that makes compileEGMf slow, but neither is really that slow by itself just translating the memory layout of Java to C++ kills it. Anyway we already know I did a partial CLI that was working for ENIGMA with GMX but I am certain it has broken since then though Harri expressed interest in developing it further. It is also important to mention that the C++ CLI itself would drive ENIGMA through the compileEGMf as it does now, so even if LGM used a CLI, compileEGMf would still be involved, the only difference is that disk I/O would be used to pass the project instead of JNA.

Basically, we've gone down every avenue of possibilities and I personally have no preference I'm just around to code whatever somebody wants me to code. I am actually working on a new LGM release fixing a number of small bugs that I hope to make available by Christmas. What the big game changer is going to be is NaturalGM. DaSpirit has continued developing it and I've been helping him. I mentioned this also in the other topic. He's taken advice from Josh who works at Google and Rusky and some of my own ideas as well. I want to keep helping him work on it because I think it would be nice to have an IDE that is maintained by somebody outside of the ENIGMA project. DaSpirit has no real interest in ENIGMA or even GM but he does enjoy developing the IDE we've been working on and does want to make it available for GM and ENIGMA. The design of NGM is very good, it follows the model-view-controller architecture for software. This means that things like project loading and serialization are part of the model. They are self contained in a library and not dependent on any GUI code. So basically you could write a C++ console program that loads and lets you edit the project with just this library.

This solves a lot of the LateralGM problems for many reasons. Because the data/model is separated from the view/GUI components you can take the library and swap the GUI out for wxWidgets/Qt/Windows Forms/GTK/JavaFX/Java Swing or any GUI framework you want. LGM was not really designed this way, though Swing is supposed to be MVC, everywhere you look in the source code our data is tightly coupled with the GUI code. So in other words, porting LGM to use JavaFX even would be a total pain because I would have to rewrite a ton of the code for the tree model and everything. With NGM you can also swap out a plugin for a room editor with an entirely different room editor. We've been making good progress too, I've helped him build a few models including one for the log table which displays output messages. We're focusing on the core design rather than just replacing LGM. And since it's primarily led by DaSpirit who has an interest in making it available to GM users as well, it means less work that the ENIGMA developers have to do. Since I ended up having to maintain LGM because nobody else would; I mean except egofree, it's just a lot of work.

So I am still going to keep developing LGM just for fun and to fix small bugs because they are easy. I am also looking to cleanup the build system so egofree can build it again after the next release. I do not think JavaFX is mature enough yet to port LateralGM to it but I will continue to assess the potential. I will also be continuing to help develop NGM because I really like the potential and I think it is the best thing to contribute towards ENIGMA. I am also open to helping you or anyone else develop an IDE if you have specific questions and just need to know how to interface with compileEGMf or anything like that. I am available and you can always feel free to post questions here. Please feel free to use any of my previous work, the links are all on GitHub.

[HitCoder]

Thank you Robert, that was very informative. (I might end up re-reading tomorrow when I'm less tired)

It'd be nice to see NaturalGM released soon, but it sounds like only you two are working on it, which is a shame, but I'd rather you spend a lot of time on it than to rush it and release it early. I have seen screenshots and it looks very nice, and the idea of being able to make your own editors for each item inside the project is pretty interesting - I'm assuming that's what you meant when you mentioned swapping out a room editor for another one.
Yes, I did think making LGM better in the ways I feel it should be made better would be very difficult, and from what you said I'd rather you not feel you have to do it just because I dislike LGM to an extent with the main reason being Java in summary.

I think it would be difficult to hook up a project in Enigma to the enigma compilers etc, as you implied. I think I may just make something in Enigma as a concept for what I want, but NaturalGM is pretty close to my idea for the IDE, I just assumed the project was dead due to some rumours somewhere.

Thanks for the reply, apologies if any of what I said is incorrect, I'm tired, it's 2AM where I am and I think I will probably get some rest any minute now.

~HitCoder

P.S: If there's a beta of NaturalGM that I would be allowed to use, or to test/try, I'm interested in doing so.

[Goombert]

Haha alright HitCoder I understand. This was not an attempt to discourage you and I would gladly help you with hooking any project up to ENIGMA, keep in mind that this is how I got started working with ENIGMA when I had no idea how it would be done. But at this point with the design of NGM we've not just tried to make a better IDE with swapping room editors and such, the point is that this architecture actually makes the IDE more sustainable in the future, which is what we need the most. We don't need another IDE like LGM or it just turning into a mess, this is not me criticizing your attempts, it's actually a criticism of my past self just thinking I can fix all of the problems by porting it to another platform/language/framework. With a big fat/monolithic program like LGM, once you've built it big enough it gets really hard to extend things because it's all glued together so tightly. MVC makes it where changing things doesn't break everything else in the process.

Also if you still have the urge to do an IDE, just do it anyway, it doesn't matter if people end up using it. Be creative with it, there are plenty of people who complain about the user interface of both GM and LGM. Try to create something original and experiment, you might just come up with something interesting like I did in LGM for the non D&D mode. If it's good enough and you can motivate me to do it or want to take a crack at Java we could even get it into LGM. Even simple things like changing panel or editor layouts (which I'm doing a bit of myself in this release I am working on). I'll tell you a better way to learn game design than using GM or ENIGMA, it's by building your own GM or ENIGMA, I can't tell you how much I've learned working on these projects.

I can promise you I will notify you and everyone else when NGM reaches a point that we can show it off without being all smoke and mirrors or vaporware, thus why we've been keeping it hush hush more than the other projects I've done just as experiments. And don't worry about anything you said, I read it all and I agreed with just about all of it, you're very reasonable and have a good understanding of the problems.

Edit: This may seem counter to what I've said and it kind of is but it's definitely worth noting, please see this wiki page:
https://enigma-dev.org/docs/Wiki/Command_line_interface

[TheExDeus]

I read somewhere that one of the developers/contributors would like to see an IDE developed in Enigma; this implies to me that it would be possible, and if it's possible, I would love to get involved.

That was my idea and I think it is certainly possible. We already have about 95% of functionality required for something like that.

Robert, I see that you have basically remade LGM 5x over the past few years and it would have been awesome if you actually managed to finish one. I was the one rumoring that NGM is dead because the repo doesn't have any commits for the past 2 years. I guess you just keep it private (although that requires GitHub subscription).

CLI is the easiest way to make a custom IDE compile a game with ENIGMA. That is why I encourage finishing it, because then HitCoder and other people who want to make an IDE wouldn't have to waste time somehow interfacing with ENIGMA. They would just need to make a writer for EGM which is a trivial format.

And talking about EGM - There are still two things troubling me and I don't think either of them are complicated to finish:
1) Change room data into XML. Hell, eyaml if you have to. Just don't do binary. Copy-paste GMS even. EGM sadly doesn't have version information inside (which is also an idiotic "design decision") so EGM would break with this change. But I don't really care. Just add version info and if there is version info load XML and if there isn't load binary.
2) Make a format that is basically extracted EGM. Just like GMX this is required for version control. Right now I am making a GUI for one of my projects in ENIGMA and have it versioned with git. But it always show up as "binary" even if I do the gitattribute stuff "*.egm diff=zip". That apparently works with docx, but doesn't work for ENIGMA.

edit:

Don't take this the wrong way, but I actually do not agree with the complaints about LateralGM on Windows because of this. Sure LGM may not be designed well for a good plugin, but its problems almost always crop up when used with the ENIGMA plugin on Windows. LateralGM by itself rarely if ever (I've never seen it) segfault on its own all by itself, it's always the ENIGMA plugin crashing it. And this is not just because of LGM's poor plugin architecture it's the generally bad design of the ENIGMA plugin itself.

It doesn't really matter what crashes. When the user opens LGM and it segfaults then of course the only thing he can blame is LGM. That is why the bugs need to be fixed. I have noticed the crashes also happening in different frequencies on different PC's. On my laptop my plugin fixes made crashes almost non-existent (about 1 crash every 3 hours or so). On my home PC on the other hand I get about 4 crashes out of every 5 times I want to compile a project, thus rendering LGM useless.

[Goombert]

Robert, I see that you have basically remade LGM 5x over the past few years and it would have been awesome if you actually managed to finish one.

I can't, it's way too much work, and nobody ever really took an interest in working with me on the projects or making them a reality except Josh offering occasional help. It's not like there's any actual architecture to them either, for the most part it's just a recreation of the GUI. That's not that bad though because they do serve as great templates/starting points and that's why the source code is still up on GitHub.

CLI is the easiest way to make a custom IDE compile a game with ENIGMA. That is why I encourage finishing it, because then HitCoder and other people who want to make an IDE wouldn't have to waste time somehow interfacing with ENIGMA. They would just need to make a writer for EGM which is a trivial format.

Yes it certainly does make it easier to get started, but ideally in a perfect world you still need compileEGMf to pass resources that have been opened but not saved in the IDE. This is the problem GM Studio's IDE has, it always forces you to save your changes before running, which 99% of the time nobody wants to do, Especially me, you make small changes just to test them. I don't even like how Eclipse or Visual Studio make me do that, but they generally get a pass because I only have the tabbed code editor open. You can't do that if it makes you save every single time. Also if the resource was already being edited and loaded in memory, it's still quicker to just pass it by memory than have the CLI read it from disk. The only time the CLI is useful, besides making it easier to interface for a new IDE, is when the resource was never loaded by the IDE. So basically it's virtually worthless because LGM was never redesigned to handle opening single resources, it still opens up the entire binary blob because of GMK, even for the EGM format and GMX too, when it could do it the way Studio and Visual Studio and every other IDE does it.

Though as I said, VS and Eclipse would both benefit by having the ability to pass the text files with changes already in memory instead of asking me to save, it would in fact be more quicker to translate the memory than do a disk write and then a disk read. Though large architectures like that would have some problems with build systems. I should also mention that it was Josh who described to me the benefits of having something like compileEGMf. It also enables the ability to do things such as live code editing, where as when you make a change to a script it automatically recompiles and is reflected in the game. You may have seen this before with online shader editors for GLSL and stuff that feature live compilation.

But yes the CLI should be finished and it would be great for supporting third party projects as you suggest and allowing them to more rapidly interface with the engine. I do not have the time to finish it Harri and where I last left it, it was compiling GMX projects.
http://enigma-dev.org/docs/Wiki/CLI

1) Change room data into XML.

I was going to do that, but clearly if that's all you want then there is GMX. XML is a bloated markup language and Josh already proposed an EYAML format to replace the binary room blobs. Anyway, I don't have time to do it, that's a substantial amount of work. The breaking change is fine though because I have backed up every old plugin and LateralGM version in multiple locations and they aren't hidden either, you can find them easily on the Wiki and I've linked it multiple times. So if you'd like to make the changes to the plugin, knock yourself out it needs done but nobody has time right now.

2) Make a format that is basically extracted EGM.

EGM is actually supposed to support this, it was never finished and I never had the interest in doing it. Also note the above comments regarding LGM still loading the entire project instead of just the resource you are currently editing.

It doesn't really matter what crashes. When the user opens LGM and it segfaults then of course the only thing he can blame is LGM.

Irrelevant, we should try to inform the user of ENIGMA's architecture, it's not difficult. Also, the plugin installs its own error handler using the LGM interface which prints the very name of the thread. The plugin, in fact, changes the URL for the "Submit Issue" button to point to ENIGMA's plugin repo instead of LateralGM's. I also made it change the message to specifically tell you that the error occurred in the ENIGMA plugin. I did that on purpose to stop ENIGMA related issues from being filtered onto LateralGM's tracker, though it didn't always work at times because some still trickled through. This is sometimes caused by a plugin issue propagating back into LGM, ie by screwing up the tree model which doesn't result in an error until the user clicks something. At the point the issue occurs in the main Swing application dispatch thread which is owned by LateralGM's process.
https://github.com/enigma-dev/lgmplugin/blob/c9039953c8a56c41b3766116fc5f3a6b801c103e/org/enigma/EnigmaRunner.java#L110
https://github.com/enigma-dev/lgmplugin/blob/c9039953c8a56c41b3766116fc5f3a6b801c103e/org/enigma/messages/messages.properties#L3

On my laptop my plugin fixes made crashes almost non-existent (about 1 crash every 3 hours or so).

So push the changes to GitHub. I do not have a working copy of ENIGMA right now because I last left off with trying to build a 64bit version. I don't have time to set everything for that up right now. The plugin is under the enigma-dev domain and again, we have all of the old versions backed up.

Anyway, I have made some time for some small LateralGM improvements and quick bugfixes I've made. I've fixed like the newest 3 issues on its tracker. I also added a new color picker control that lets you copy and paste hex color values. I am doing this release to fix the JoshEdit problems so that egofree is able to build LGM again and people can continue to develop it. Then I have to get back to doing some other side hobby projects. I also added the right orientation mode as a preference and it's implemented in every editor. There will a bunch of tiny other changes regarding improvements to some of the layouts and stuff as well. I know in this screenshot the layout looks bugged because I'm still working to get everything working nicely.

[egofree]

I am doing this release to fix the JoshEdit problems so that egofree is able to build LGM again and people can continue to develop it.

Thanks Robert, i really appreciate !   

[TheExDeus]

Yes it certainly does make it easier to get started, but ideally in a perfect world you still need compileEGMf to pass resources that have been opened but not saved in the IDE. This is the problem GM Studio's IDE has, it always forces you to save your changes before running, which 99% of the time nobody wants to do, Especially me, you make small changes just to test them.

That is really not a problem as you can just write to a temporary. Saving a project usually takes a second or two - just make it save to a temp directory and compile that. It is a lot easier to make saving and loading faster for CLI than to make interfaces between IDE and ENIGMA (as LGM shows).

I was going to do that, but clearly if that's all you want then there is GMX. XML is a bloated markup language and Josh already proposed an EYAML format to replace the binary room blobs. Anyway, I don't have time to do it, that's a substantial amount of work. The breaking change is fine though because I have backed up every old plugin and LateralGM version in multiple locations and they aren't hidden either, you can find them easily on the Wiki and I've linked it multiple times. So if you'd like to make the changes to the plugin, knock yourself out it needs done but nobody has time right now.

I don't see how that is a lot of work. In C++ I can do it in a day. The problem is that LGM is made in freaking Java. And we did have posts with Josh talking about rooms in a non-binary mode. In it I showed how XML is actually not bloated if used correctly. Yet has more features than EYAML proposed. But I honestly don't care in which format it is. Use a CSV or Excel if you have to, just don't make a version breaking incompatible binary blobs.

EGM is actually supposed to support this, it was never finished and I never had the interest in doing it. Also note the above comments regarding LGM still loading the entire project instead of just the resource you are currently editing.

Again, if you have EGM loading then this should be trivial. It should be even easier, as you must SKIP a step. Just don't open the .zip and instead read the folder structure.

Irrelevant

What you said irrelevant. Users don't even differentiate between LGM and ENIGMA even if you change the title bars. This site is called ENIGMA-DEV having a game development tool named ENIGMA. When you install it and run it (on windows you run ENIGMA.exe) you can make the logical assumption that the editor is also ENIGMA. And the errors you mentioned are Java exceptions - they are not the problem now. It is quite easy to create a Java exception by changing .ey files or by not having correct paths because LGM/plugin does 0 error checking, but if the plugin crashes then there is not exception. Just crashes like a regular SIGSEGV. Hell, in Win98 or even XP ENIGMA would probably trow BSOD's with the current stability.

So push the changes to GitHub.

I have. All my changes are in my branch. But there must be a million more. The whole .dll needs to be rewritten to get rid of the unsafe code.

I am doing this release to fix the JoshEdit problems so that egofree is able to build LGM again and people can continue to develop it.

Good to hear.

[Goombert]

That is really not a problem as you can just write to a temporary.

While writing to the disk is certainly not as expensive as it used to be, it also simplifies the design in addition to enabling the other features I mentioned like live recompilation.



The above diagrams should help convey this a little better. If you use purely a CLI it severely complicates the overall process of passing the resources from the IDE to ENIGMA where it eventually has to be passed to compileEGMf anyways. The top diagram is what we get when we use a plugin for the IDE view/presentation tier. The bottom is what we get when you use only a CLI. The CLI is meant to serve as a replacement to the presentation tier only and not really support another presentation tier on top of it. This significantly complicates the overall process of passing the resources by requiring them to be loaded into the IDE, written to the disk, then read back from the disk on the other end before being passed from the CLI to compileEGMf. It is much simpler to have a plugin to just pass what was already loaded in memory directly to compileEGMf as we do with JNA.

In the model-view-controller architecture, LateralGM and the CLI are a view, they are simply different ways of viewing and manipulating the data.


For a managed application like LateralGM, the CLI is only useful for telling compileEGMf to find resources we haven't loaded. But it's not useful to LGM right now, because LGM loads every single resource whether you are editing it or not. In fact, the correct design would have the CLI never even referenced by the IDE, compileEGMf is missing the functionality of finding resources and loading them from disk which the plugin should also be utilizing. This is a significant amount of work though.

However the CLI does make it extremely easy from a conceptual stand point to build a new IDE and get ENIGMA to compile the project files, this is not debatable and I'm not arguing it. A lot of people also really like a simple command line and this is partially why I created it, even though I also created it originally because of my naive understanding of how it could just simplify everything and I thought we'd no longer need a plugin (which is not true as I've described). So yes, by all means the CLI needs finished but I do not have time, and it would be perfectly ok to use it for a new IDE until that new IDE has a proper plugin framework.

For reference, you can read why IsmAvatar originally invented the plugin in the first place, and it's for the exact reasons I mention.
http://enigma-dev.org/docs/Wiki/CLI

Enigma started out as a CLI exe only. LateralGM was forced to communicate to it through this limited fashion, writing the game to a file, passing the filename to the exe, and pipe-reading ENIGMA's output from the exe and files that it produced, which was much slower than it had to be.

One day, IsmAvatar figured out how to get Java and C++ to communicate via DLL with the use of JNA, which allows LGM and ENIGMA to share memory structures and make interactive calls to each others functions, which is much faster and much more dynamic. Since then, ENIGMA has redirected itself towards being a DLL, and the build process has been streamlined for that purpose. As a result, ENIGMA became almost exclusively a GUI program.

Due to the thee-tier architecture of LateralGM (front-end back-end separation), it isn't hard to use a headless LateralGM - that is, just the data and logic tiers, without the front-end presentation tier. As such, it wasn't long before this fact was exploited, and combined with the Plugin to communicate with the dll, to construct a Java CLI for ENIGMA.

And we did have posts with Josh talking about rooms in a non-binary mode. In it I showed how XML is actually not bloated if used correctly. Yet has more features than EYAML proposed. But I honestly don't care in which format it is.

Rusky has pointed out to me that XML supports namespaces and some other features I see, so excuse me for being naive originally. I was not aware of this but I think it is still debatable about how verbose XML can be, and I clearly tend to agree that it is. However, XML apparently does have some good data processing and querying features (which I was already aware of), but why exactly does ENIGMA need them? YAML is clearly/technically/colloquially not a markup language and really good for just plain old data, which our rooms basically are. So I would like to know if you had some ideas for utilizing the XML processing/querying features or not, because that is something to consider before making a decision.

What you said irrelevant.

I don't know what you want me to say. LGM has taken all of the necessary precautions to make sure the errors are properly reported, and so has the plugin framework. Even a native IDE would require a plugin framework so that it's not limited to only ENIGMA, it's a simple separation of duty. What is the problem here? Are we still not able to debug these compile errors you're having with the plugin or what? I don't know how they've managed to creep up all of the sudden. The plugin was unstable but not unusable when I last left as it appears to be now.

I don't see how that is a lot of work.

Because I'd have to set up ENIGMA and the plugin which are all broken locally because again I was messing with 64 bit. Not to mention you have to go and regression test and make sure it completely works the way it should, file handles are properly closed etc etc. Additionally I haven't worked as much with the EGM format and tried to leave it alone for the most part besides adding UTF-8 and closing file handles so the project could be deleted once the IDE finishes with it, it was causing tons of problems before with saving EGM's when I first got here, I eliminated those issues a long time ago. EGM was really unfinished and still is pretty unfinished though we've added timelines and some other things.

Again, if you have EGM loading then this should be trivial. It should be even easier, as you must SKIP a step. Just don't open the .zip and instead read the folder structure.

Yeah it probably wouldn't be too hard but it would need tested and you'd have to double check and regression test etc. I am not up for it because I don't have time to set up all of ENIGMA and everything right now. I have like 1000 other projects I am wanting to be working on and my school work, I'm still going full time.

[TheExDeus]

This significantly complicates the overall process of passing the resources by requiring them to be loaded into the IDE, written to the disk, then read back from the disk on the other end before being passed from the CLI to compileEGMf.

It doesn't complicate anything as the IDE must be able to save EGM anyway and the CLI needs to be able to load it. If you have these two features then this interface is as simple as:

Code: [Select]

functionThatSavesInEGM(randomlyGeneratedFileInTemp);
system(cliPath+" -egm "+randomlyGeneratedFileInTemp);Normally you would do some checks here but this is truly is as easy as that.
I do understand the positive side of using "compileEGMf", but "less complicated" is not one of them. It's the contrary. I also know the negative sides which are more. And I do support compileEGMf existing and a plugin existing, but only if they are stable. That is why I think we need to finish CLI and hook it up (maybe optionally) with LGM. So I can at least use ENIGMA while the plugin gets fixed. I feel having a stable CLI to be easier to achieve than having a stable plugin. Although technically the CLI still uses the compileEGMf so it might be just as unstable. But a lot easier to debug for sure. Many crashes in LGM happen even when nothing is clicked (just minimize and maximize the window a few times). At that point the only work LGM should be doing with the plugin should parsing headers - thus reducing the places we should be looking for a fix. I think the parser is the main error magnet here.

Rusky has pointed out to me that XML supports namespaces and some other features I see, so excuse me for being naive originally. I was not aware of this but I think it is still debatable about how verbose XML can be, and I clearly tend to agree that it is. However, XML apparently does have some good data processing and querying features (which I was already aware of), but why exactly does ENIGMA need them? YAML is clearly/technically/colloquially not a markup language and really good for just plain old data, which our rooms basically are. So I would like to know if you had some ideas for utilizing the XML processing/querying features or not, because that is something to consider before making a decision.

I do agree that XML is more verbose than some alternatives, that is for sure. But I don't really see an issue with that. It matters if the XML must be read by a user or the XML can make size a little larger which maybe was a big problem 15 years ago. But in our case the XML is only for the IDE and the CLI/plugin (which means the XML could look like "<093420uf09ihsudoifhirwouh97 au='sqqd1'></093420uf09ihsudoifhirwouh97>" if we wanted) and the size in the modern world is no longer a problem. For an XML file to even be 1MB in size it would have to consist of A LOT of information.

I don't know what you want me to say. LGM has taken all of the necessary precautions to make sure the errors are properly reported, and so has the plugin framework. Even a native IDE would require a plugin framework so that it's not limited to only ENIGMA, it's a simple separation of duty.

I didn't mean the error reporting is bad, I just meant that the excuses like "Plugin is crashing, not LGM" is futile here. When the plugin crashes, LGM crashes. The only thing I don't like about LGM's/plugins error handling is that there aren't that many checks done. Removing one .ey entry can trow a segfault. I think one of the main reasons for a markup language for settings is that you can check if the settings exist and use the default if they don't. Or at least report in a sane matter. But it seems EYAML wasn't made with this in mind, as I actually had to add a function to check if an entry exists myself recently "bool ey_data::exists(string n)"  (https://github.com/enigma-dev/enigma-dev/commit/fc5a90a4b8e64718927fb2765c5ee4572ee3ef80).

Are we still not able to debug these compile errors you're having with the plugin or what?

This is a problem, yes. I personally cannot get any backtraces in 90% of crashes.

For EGM I think the first step is to write a real specification. I will look in the http://enigma-dev.org/docs/Wiki/EGM and http://enigma-dev.org/docs/Wiki/EGM_format pages and see if I can make something better.

[Goombert]

It doesn't complicate anything as the IDE must be able to save EGM anyway and the CLI needs to be able to load it.

You've got the idea now, the assumption that the plugin could be abolished, I admit, was originally me proliferating my own stupidity. Now let me beat the point home a little harder.

It's simpler from an external perspective, but not from an internal one.

Externally it does look like this...

Code: [Select]

functionThatSavesInEGM(randomlyGeneratedFileInTemp);
system(cliPath+" -egm "+randomlyGeneratedFileInTemp);

But internally, throughout the whole process, it turns into this...

Code: [Select]

functionThatSavesInEGM(randomlyGeneratedFileInTemp);
system(cliPath+" -egm "+randomlyGeneratedFileInTemp);
nativeCodeThatReadsEGM();
cliPassesDataToCompileEGMf();
compileParseLinkAndStuffs();

Instead of this...

Code: [Select]

pluginTellsCompileEGMfWhereToFindTheRemainingResources();
pluginPassesDataToCompileEGMf();
compileEGMfFindsResourcesTheIDEDidNotHaveLoaded();
compileParseLinkAndStuffs();

Note the final example is in a perfect world where LGM had the proper architecture, GMK was not a binary blob, and compileEGMf was able to load/find resources. Anyway, yes, go CLI, there are numerous benefits for it. It also serves as an excellent tool for when LGM is just really failing with the plugin, and it also loads and compiles projects quicker if you do not need to change anything at all.

I do agree that XML is more verbose than some alternatives, that is for sure.

I don't know, reading the XML is useful when a corruption occurs, that's why we don't want binary blobs. I wouldn't say XML is hard to read, I did HTML programming in junior high school. I don't know how I would have been towards YAML when I was younger. When I first saw it a few years ago in ENIGMA I wasn't sure I understood it, but it wasn't difficult even though it looked foreign. Now that I am older, I definitely prefer it just because it's easier on my eyes. That said, GMX is basically the XML format, we've done YAML in every other part of EGM. So that is basically why I don't support the room format being XML because that is inconsistent. Using YAML makes the EGM format unique from the others, and I really don't think we should do another format just because we want XML. There's not really a good argument at all here especially considering we already have the suitable YAML infrastructure.

I just meant that the excuses like "Plugin is crashing, not LGM" is futile here.

Sure, but I've not been saying that. I've been saying, stop looking at LGM, and look at the plugin. Specifically because several of the issues I am already aware of and think I know what's causing a large bulk of them. A lot of them being related to the changes in Java 7 that made plenty of Swing apps start showing errors about LegacyMergeSort. Those I believe are related to doing GUI stuff when not on the Swing Event Dispatch Thread because Swing is not thread safe. I never understood it when I was younger but it basically means doing GUI stuff on any other thread leads to unpredictable results. The plugin gets a lot of these because it makes a lot of use of threads and tries to do GUI stuff from those threads. A few versions back I tried to wrap a bunch of stuff in SwingUtilites.invokeLater but I am not sure what benefits it had.

You can search all over the internet, these have been creeping up for plenty of Swing apps:
https://community.oracle.com/thread/2457538?start=15&tstart=0

Anyway it's really not a Swing problem and quick hacks to enable the old legacy sorting should not be used. You have to ensure this functionality in any non-thread safe GUI framework, even though Swing is a heaping pile of garbage. The real correct fix is to analyze existing code and make sure GUI stuff is being done on the main Swing EDT. I can't take a look at the plugin right now but I did fix these issues LGM side where the project loading is threaded many versions ago. It's important however that when you guys experience these issues you not just file a bug report with the stack trace but tell me exactly what you were doing. It's impossible to track down, as I have never found a way to reproduce it consistently, because the code is being executed on discrete threads.

I haven't spoken up about this because I thought it was known after my research with egofree. This is why I continue to say the plugin was designed poorly because nobody considered thread safety. Also remember that EGM used to leave a ton of open file handles. So not everything in that plugin was finished and I patched around a lot of it not knowing what I was doing. However, I think the effects of anything I've done are not that big, it clearly seems to have been getting worse even though I haven't been touching it. So my analysis and research has supported my conclusion that it is the plugin that was designed poorly and old issues are creeping up to bite us in the behind some more.

I think this ticket is one of them, I'm trying to get more info from the person, these tickets always really annoy me when people don't respond:
https://github.com/IsmAvatar/LateralGM/issues/218

Removing one .ey entry can trow a segfault.

Yeah there should be some more user friendly error reporting that reports that, possibly by integrating it with the compiler's existing error reporting stream/interface. It could actually just show up and return back to LGM in the console window, that's not even a plugin issue though, that's an ENIGMA issue because the same thing would likely occur in the CLI.

[TheExDeus]

But internally, throughout the whole process, it turns into this...

But the last three lines run in the CLI, which can read EGM's and compile the game as it is its purpose.

I don't know, reading the XML is useful when a corruption occurs, that's why we don't want binary blobs. I wouldn't say XML is hard to read, I did HTML programming in junior high school. I don't know how I would have been towards YAML when I was younger. When I first saw it a few years ago in ENIGMA I wasn't sure I understood it, but it wasn't difficult even though it looked foreign. Now that I am older, I definitely prefer it just because it's easier on my eyes. That said, GMX is basically the XML format, we've done YAML in every other part of EGM. So that is basically why I don't support the room format being XML because that is inconsistent. Using YAML makes the EGM format unique from the others, and I really don't think we should do another format just because we want XML. There's not really a good argument at all here especially considering we already have the suitable YAML infrastructure.

YAML is not a pretty way to show large amount of data though, as it mostly useful for settings. That is why for objects we already use a different format called "EEF" (http://enigma-dev.org/docs/Wiki/EEF). Basically we invented like 3 "standards" just because we didn't want to use existing ones. Adding a third language to EGM files would be unfortunate, so maybe I can look into the EEF stuff as a way to encode rooms. I think a while back in the room topic we talked how EYAML was actually ugly for room data, but I'm not sure. I will have to look up that topic.

The plugin gets a lot of these because it makes a lot of use of threads and tries to do GUI stuff from those threads.

Well maybe making it only use one thread is solution then? It certainly shouldn't do GUI stuff on another thread as there isn't any gains from it. The resource passing can benefit from threading, but then of course it must be made safer by at least using only one thread.

[Goombert]

Idk what else to say, both the CLI and compileEGMf are needed to get the best results. Anyway, you have my opinion on mixing XML with YAML for EGM, I was still interested to know if you can think of some way we could leverage XML's querying abilities in a unique way. Those wiki links are the best place to look for the other EEF formats and what not.

Well maybe making it only use one thread is solution then?

That's the problem actually. For example the output window overflows sometimes because its reading a stream of text from the compiler and updating the GUI off the Swing Event Dispatch Thread. Because Swing does other things when not on the EDT, updating the GUI component by setting its text is inherently unsafe when your not on the EDT and leads to unexpected/undefined behavior as the official Java documentation will tell you. This is again, a problem with many GUI frameworks that are not thread safe, just like Swing. I believe Windows Forms is not thread safe either, however the new JavaFX is thread safe. Swing is pretty legacy these days, if you look at its source code its like halfways undocumented and doesn't mark barely any of its methods final. JavaFX is a much cleaner solution, but I'd sooner rewrite a new IDE with it than try to port LGM to it. I think WPF is thread safe like JavaFX because they both support data binding.

[Josh @ Dreamland]

I believe it's time to invest in replacing LGM. LateralGM follows the three-tier architecture, which is different from what Robert has linked but which does mean that there should, theoretically, never be a reason to replace it. As code from any layer becomes obsolete, it can be replaced as needed to keep current. However, due to some neglect, and some happenstance, all three tiers are obsolete.

• The presentation tier is obsolete. Swing is a lousy framework with long-lived problems that developers refuse to correct. The inability to support mouse horizontal scroll is an example. Not to mention, people prefer native widgets. This isn't entirely out of the question in Java. Either way, this tier should be scrapped.
• The logic tier is mostly solid, but makes fundamental assumptions of how the data tier interacts with data. Specifically, it assumes that passing all resources to the data tier will accomplish the task of writing out a game file. It offers no way of detecting whether a resource has actually been modified, and changing this is a TON of work. I think Robert once tried to invest this work, but I'm not sure if he ever got anywhere. By a "ton of work," I mean enough that recoding this tier correctly, from scratch, is not out of the question, even if we stuck with LGM. Game load times are similarly affected; the load mechanism requests that all resources are loaded up front, which includes massive MP3 files and long animations. If meshes were to be supported, they would also have to be loaded up front. This is a huge problem for LGM's progress into serious game development.
• The data tier makes, of course, the same assumption. It serializes all data at once, doing no change detection, which means that saving a GMX/EGM document takes the same amount of time as saving the equivalent GMD/GM6/GMK/GM81. Actually, it takes longer on Windows due to filesystem overhead. Saving a document with only one changed resource should take virtually no time for GMX/EGM, and the entire game time for GMD/6/K/8/81. The same problem applies to game load. For later, I'll mention that five different people, Robert included, have already ported all relevant pieces of this tier to other languages.
• All three tiers are written in Java. A lot of users have issue with Java, and 75% of our incoming tracker tickets are from people using the wrong JVM architecture on Windows.

With respect, I believe that people who choose the second option do so from a usability standpoint, unaware that their few problems with the IDE require an entire recode, in Java or in any language.

I forward that since a huge recode, even if not a complete recode, is required for the further progress of the LGM project, this recode should be done in a framework that does not tow the JVM behind it, and preferably one which uses native widgets. Sound (cogent) logic—and there's a lot of it; I know because I wrote some of it—should be preserved or ported, and the rest is being scrapped, anyway.

[HitCoder]

I'm just bumping this to ask a question;

Would it be possible for me to make an IDE within Visual Studio Express? I've recently applied to a college and will be learning C# at the start of my course, I plan to get a degree in computer science later in life too, but I'm starting to learn some things with VS Express already, and since I have no problem with the LateralGM IDE on Linux, I won't need to make a custom IDE for Linux. Don't ask why, it just feels a lot cleaner on Linux, but Linux doesn't work with things like Skype properly so I have restrictions to how I can do things.

Anyways, TL:DR, I would like to know if there's anything stopping me making an IDE in MS VS Express.

Thanks again.

~HitCoder

EDIT
I noticed SharpGM was made in C#, and the interface looks like it was made in Visual, so I apologize for asking a stupid question. Thanks anyways =P