Google Summer of Code

[Josh @ Dreamland]

Hello, everyone; long time, no see.

Google Summer of Code signups are around the corner. Are we, as a group, interested in participating? I am considering sponsoring either ENIGMA or NaturalGM, or even both, depending on which project is able to generate an workable list of tasks for contributors.

The problem with ENIGMA's participation is that its contributor base is already mostly college-level help, so the majority of tasks that would be easy to delegate to new help are already completed, while the remaining work requires more skill and background info on what ENIGMA is and what it is supposed to be.

That said, looking over the wiki page from last year, some of the workpoints seem nearly feasible. The problem is that no one has started any of them. If we want help, we have to be serious about having room for participants to actually contribute code. We can't be allocated a new contributor and expect them to figure out what ENIGMA is and just start tacking entire new systems on. Work points such as "Implementation of an Asynchronous Networking System" are entirely too much work for someone who is unfamiliar with the project. Does the Network_Systems folder even work like the other systems folders? How will a new contributor even know they are on the right track?

If we are to participate, we need tasks that begin with ten lines of code and end with implementing an entire system. There needs to be room for a contributor to start small and grow in the project. I can't endorse ENIGMA with such course-grain tasks. This is the reason I stayed out of the GSoC planning last year, and if we can't generate a better task list this year, we'll hit the same fate.

We can improve the "Implementation of an Asynchronous Networking System" task by creating a reference list of what we actually expect this system to look like, and offering an example game which, given these functions, should be able to be played over a network. We can't expect a new contributor to write the game that tests his or her new functions. That's way too much work for someone who has never heard of the project.

Related: Do we even have a simple dev wishlist, anymore? I occasionally hear about X huge thing that is completely broken and everyone has been working around, but I have essentially no idea what people want to see out of this project right now, aside from a new compiler. It seems to me that we have no little tasks; the project is waiting on someone to "just go through and fix everything." Does anyone care to prove me wrong?

To be brief, if we want to participate in GSoC, we have to make sure we have tasks that are actually suited to someone who has never heard of the project. Otherwise, I guess we can sit this one out.

[Josh @ Dreamland]

The rebuttal I'm receiving is that it is the job of the mentor to provide guidance for the recruits. This is certainly the case, but we can't postpone things like authoring test games until the student has already created code. We should have a sample game or other test suite to give the student as soon as they join. At very least, in the case of a networking system, we should have a game that is working except for the networking bits, which can be broken, commented, or completely missing—as long as there is minimal effort to adding them in. We can't have our mentors scrambling to write a buggy test game while the student hangs around waiting. That would be a waste of the student's time and Google's money.

And yes, students are suppose to be somewhat autonomous and generate their own work items—this does not mean that we can leave the mentor and student to completely improvise the entire process. Some real planning ahead of time is still required. At very least, students should have a breadcrumb trail to follow in the absence of the mentor.

Here is an interesting news bulletin: I am allowed to mentor as a Google employee, and current contributors who are also students are allowed to sign up.

[daz]

Frankly, I don't think anyone else around here is knowledgeable enough to be a mentor other than you, Josh. Hell even the last time I tried to do something big to Enigma (Android), that was 99% you. And I just had to give up, because there was no starting point for "hey yeah this is how these systems work together and connect to the engine core", plus my C++ is hilariously weak to begin with [and when you weren't around, which was most of the time I just couldn't progress which made it too frustrating to continue]. Hell all of that work is now lost to the ether after I moved to an SSD, the only remnants are some vague instructions I left on the wiki which assumes the user has a copy of an Android fork. Maybe I should have used Github for some poor unfortunate soul of the future. I'd say there needs to be a lot of documentation done if you want someone to ever get up to the system or platform level. This project is a relatively undocumented mess that is pretty hard to jump into. And yes, LGM is a huge pain point as well. I'd be in favor of putting another IDE for GSoC instead, because I would like that option.

About networking: last I tried, Berkley sockets worked. What networking are you even looking to get done?

As for feature requests and what not, those are a dime a dozen. Enigma is missing an assload from GM still for starters.

[DaSpirit]

Tasks such as "fix this" or "add these few functions" are not good tasks. We need to think of large tasks that would take three months to finish for someone who is completely new to intermediate to the topic.

[Josh @ Dreamland]

Dime a dozen tasks are great. They're a great way to get a new contributor interested in the system. I think we'll hit a snag trying to sell some of the larger tasks to people. Keep in mind, the candidates are going to be paid, but they still get a say in the project they work on. We're competing with how interesting a task we can generate.

So, can anyone list any of these "dime a dozen" tasks?

[daz]

I'm going to just list a bunch of random things that might have already been completed or might be an insane amount of work. I have had a wishlist, and some of these might be up to 2 years old.

• User events. God fucking damn it I use these all the times, and these were never working when I was using ENIGMA
• Sound system improvements. I don't know if the new audio engine from GM has been implemented or not, but last I checked even sound_volume did not work for the basic system. Sound position.
• Particle systems with advanced features: attractors etc
• Views system is extremely inconsistent when compared to GM
• Pixel perfect collisions
• Polygonal collisions with collision editor
• Full box 2d integration, complete with editors and other IDE integration
• Working HTML5 export via emscripten or equivalent
• Android because god knows we failed. Most basic would be getting it to compile and open an application. More advanced might include writing a GLES2 renderer, sound wrappers, and services integration (Google Play)
• I swear to Christ it's been 3 or 4 years since you promised that new compiler. I've run into a lot of dumb things the compiler couldn't parse that was perfectly valid GML
• Force the parser to prepend identifiers to various resources like scripts, objects, rooms, and variables in code and so on so you don't get "reserved word" or worse yet, the variable already exists somewhere in the engine code base
• Game debugging.
• Game profiling
• The last time I checked, room speed was not consistent between platforms (Linux and Windows). I'm not sure if this was fixed by the changes that happened to implement a more precise timer on Windows or not. Needs to be double checked and/or fixed.
• Proper way to export games instead of having to go to temp files and copying the executable
• Async events from GM
• Configurations like GM. Allow for exporting included files for certain platforms, and allowing for easy platform switching IFDEF equivalents in code
• Texture groups support like GM

That would be a good start anyway.

edit--
Someone would need to go through the code and check what functions have been implemented and what haven't, from GM. Some of those might be easy to implement, but the wiki is often not updated when a function is implemented, or whether it is still a stub or completely missing.

[TheExDeus]

Proper way to export games instead of having to go to temp files and copying the executable

This has actually been fixed.

I only need a new parser/compiler. I don't know what else people can do, besides implement a million features. I agree that ENIGMA is very undocumented and can be hard at times, but hell, a person made a LUA extension in one day (at least compiling), so we haven't totally failed. I think we need more documentation and a new website with tutorials and stuff. We should probably go trough GitHub issue pages and fix bugs. I have actually fixed many of them, but as nobody else tested for them, the issue remained open. So I guess part of those issues are already resolved.

I guess another thing a person can do is Android support of some sort. Like daz said, the first step is getting a project to compile into apk, then the person could change either the current GL3 rendering system or create a GLES2 one, that would allow porting games.

[Josh @ Dreamland]

dazappa:

• User events. I thought polygone finally got those working. Are you sure they don't? If not, I'll consider adding them as a task.
• Sound system improvements. Robert added most of those, but he did so when he was new. No idea if they work. Worth investigating, and can certainly be a task.
• Particle systems with attractors etc Does forthevin's extension not support attractors? Tell me what's missing, and consider it a task.
• Views system is extremely inconsistent when compared to GM I'll need you to elaborate, again...
• Pixel perfect collisions We definitely have these. Are we missing rotation? What's missing?
• Polygonal collisions with collision editor Added as an IDE task.
• Full box 2d integration Added as an extension task stub. Can you elaborate?
• Working HTML5 export via emscripten or equivalent We have this. It was never hooked into LateralGM because it and the compiler are not extensible enough.
• Android because god knows we failed. Historically speaking, the reason we don't have this is because I'm bad at holding people's hands through it. I don't think GSoC will change that.
• New compiler. Good wishlist item, bad GSoC item.
• Force rename of resources with junk names. Okay, this is a small enough task that we can add it.
• Game debugging. Required metadata exists in now-bitrotted compiler branch. Still less effort to implement in a new system than current master.
• Game profiling. See above.
• Room speed was not consistent. Dozens of people have worked on this, and it's my understanding that it's finally fixed.
• Export games instead of having to go to temp files. Solved, then broken, then solved, then broken, then solved, then broken, then solved, then broken, then solved, then broken, then actually solved, then really broken, then solved, then broken, then solved, then broken, then solved (?)
• Async events from GM Don't understand the use of these; not personally invested in task enough to hold someone's hand through it.
• Configurations like GM. I know nothing about these, but IFDEF makes me think it's a parser task. The only task I'm honoring for the current parser is "SHRED AND BURN."
• Texture groups support like GM Sounds good. Also sounds mostly like a UI task (Correct me if I'm wrong).

Harri: are you currently a student? Or, are you interested in mentoring?

[Josh @ Dreamland]

I have created a new Wiki page for these tasks. Feel free to edit the page to elaborate on tasks. If you add a task, please also begin a discussion about it here. Please don't delete a task without discussion beforehand.

[TheExDeus]

Sound system improvements. I don't know if the new audio engine from GM has been implemented or not, but last I checked even sound_volume did not work for the basic system. Sound position.

I think most of them worked. What is missing is sound effects, but I don't think we can really add them to existing systems. We also have both OpenAL and DirectSound which seems redundant, as one is not really better than the other.

Particle systems with advanced features: attractors etc

Looking at the source it seems deflectors and attractors are in. I think the system was finished. The problems was with rendering, because for every graphics system a bridge was needed. As as quick fix I added PS_particle_bridge_fallback which uses General drawing functions to draw particles. This is slower than having a special rendering function, but it at least works on every graphics system. A GsoC task could be making the particle system faster, better or even implementing it on GPU vai OpenCL or something. It's "hard", but 100% completable in 3 months. Especially if nothing really has to be changed ENIGMA side and can be done by modifying the extension.

Views system is extremely inconsistent when compared to GM

Elaborate, because I fixed view_angle which was the last known bug I was aware of. There can be some more inconsistencies, but I use views all the time and haven't noticed any problems. I don't use GM though, so I might not know about something. One thing we are missing we potentially need is view_surface, which allows easier post-processing effects.

Pixel perfect collisions

That should work now.

Polygonal collisions with collision editor

Would love that.

Full box 2d integration, complete with editors and other IDE integration

Box2D should be working fine as functions, but it's not implemented in the IDE. So that can be done.

Working HTML5 export via emscripten or equivalent We have this. It was never hooked into LateralGM because it and the compiler are not extensible enough.

How "working" was it? Remember seeing it, and it had the basics right, but that didn't involve making an HTML5 project via LGM. It also involved coding JS engine from scratch, which can potentially be bypassed using emscripten.

Game debugging.

Robert started working on that and I would love this too. I actually started a discussion on how this would work, because we need to map the user code to parsed C++ code, which didn't seem totally trivial.

Game profiling

I started on a primitive GPU profiler, but it wasn't meant to be anything fancy, so it would probably not fit. It would just return how many draw calls are done, how many vertices are drawn, how many models and textures are used and so on. But no timing or GPU temperature or something like that.

Texture groups support like GM

This is actually needed. Texture groups are called "texture atlases" or can be implemented with "texture arrays" or in more recent OpenGL "sparse texture arrays". It requires a slight recode of the texture system and modifications to all drawing functions. I want this because it's most useful for 2D games and applications (which I mostly make and test). Like if I could put the font texture in the same texture page as my sprite, then draw calls would plummet in my newest project from 500 to 1.

Harri: are you currently a student? Or, are you interested in mentoring?

Currently I'm a Phd student. I'm not sure about mentoring though, as I cannot promise anything time wise.

What are you currently doing? It seems you are very time consumed with your job, as I haven't seen any commits by you in any of the ENIGMA or ENIGMA related projects.

An IDE suggestion would be custom resource support. Allowing everything to be saved in EGM (it's a zip file, it should already support everything with 100% backwards compatibility, but for some reason it's broken now). So it would be possible to add an extension, that would add a resource or even an ide to view it or edit that resource (which can be done now, as when you open sprite in your own paint program). Like I'm making a GUI system for ENIGMA. I would want a gui editor in LGM, but adding that would be painful for me. So I can make an external editor (in ENIGMA itself), that can open my hypothetical GUI save file and edit it. Then save it and LGM reloads it. And then I can use it in code. Same with shader editor. Making a shader editor in LGM is practically impossible. But making one ENIGMA isn't.

[Josh @ Dreamland]

I'm afraid it's not that simple. It isn't that I have so little free time—it's just that when I sit down to fix one of ENIGMA's problems, I find myself traversing a whole loop of increasingly large problems until I lose interest and stop working. There are so many broken things with it that I just don't want to fix any of them, because I have this feeling that it makes no difference in the long run. And the amount of time required to fix one item correctly isn't itself all that high—it's just that I only have two days a week that are all mine, and it isn't long enough to fix something completely. And when I start to fix something and am then interrupted by a week of work, I lose interest in it, and then it gets bit rot. So really, it's more an issue with contiguous ENIGMA time. Even when I do take a week off, I spend it visiting family. So...

I actually have more free time now than I did in college. The problem is focusing on ENIGMA long enough to make a difference. When I was young, I'd make a plan and stick with it even if evidence (read: Rusky) suggested it would eventually fail. This is why anything ever got done. Now that I'm older and can see around corners, it's not so easy to just continue down the path of gradual improvement.

Most of the little things in ENIGMA are done. The project right now probably doesn't really need people to do little things so much as it needs someone to finally get the time to do big things. But I figured it'd be worth bringing up GSoC anyway. The reason I created this thread is because I'm not personally convinced we should participate in GSoC, for just those reasons. But if you can convince me, I can convince Google.

If you want to know what I'm working on this moment, it's a program that reads in a video and produces MEIs of frame intervals of a given length. The reason for this is complicated and involves my landlord. Before that I was working on a new flags library, which is for general-purpose use. Before that I wrote a world editor for Terraria worlds. Before that was Christmas, and around that time, I recoded a piece of ENIGMA's compiler, and eventually did merge that in. So that's one less thing that needs done before I can actually get anywhere with it. Before that, I don't really remember.

I don't know that there's a simple answer to what it will take for me to sit down and finish that new compiler. IDE support for all the wonderful things dazappa is requesting would certainly help. Maybe I should nip this in the bud and just write a damn IDE. If I finish the widget framework I started in ENIGMA ages ago, that would probably help. But even then, ENIGMA on Windows is a nightmare. I don't know how you put up with it. I guess you're clever enough to set up your own compiler so you don't have to put up with the release zip cheeseboy started Robert compiling. The whole thing is mucky, and I just don't like touching it. And every little administrative task wastes so much goddamn time. I spent four hours today trying to figure out how to set up an SSL certificate for ENIGMA's site. I've wasted tons of time looking for a good license. The EDC's a wreck; I've wasted hours on it, too. There are things I'm not good at, which I normally end up doing, too. It apparently doesn't need done that bad, of course, because my attempts to just ignore it all have been largely successful. But then it just balls up and I don't want to go back.

I mean, if all you want is a new parser, it's in the branch. It doesn't support all the things I wanted it to, but it's correct. It will want the new JDI, which is in another branch. I went to merge those, but my lexer changes were incompatible, so I went to consolidate those so future changes would be easier to merge in. I then discovered a couple problems with the way JDI handles macros, and went to fix those. It got messy and resulted in a slowdown, so I elected to start replacing the macro system, which I hoped would solve both problems. I got distracted doing other things, including the fucking EDC and license problem, and never finished.

I imagine that gives you some idea of how I feel.

[TheExDeus]

I get that feeling. But from functional standpoint those are the only two issues I have with ENIGMA - a parser that doesn't parse valid EDL/GML and so I require workarounds, and LGM which often crashes and corrupts EGM's. And what bugs me is that both are out of my personal expertise. I can make many things in ENIGMA engine, but I sadly cannot do anything of that sort. While I'm working on my projects for next few months I will probably implement a lot more things in the GUI extension, fix up many renderings things I will encounter, add an extension for scripting language (either LUA or AngelScript) and more. And I will do that because ENIGMA is the IDE part of my project - I make a computer vision program that allows you to perform CV with nodes and real-time scripting. Everything behind the scenes will be an C++ extension for ENIGMA which can be standalone compiled. While the visual part is coded in ENIGMA. This makes me work on ENIGMA. Maybe you also need to take a project which can be done in ENIGMA and do it there? Like your Terraria editor (which I guess wasn't for Google) - It could of been coded in ENIGMA, and while making it you would probably fix some bugs or implement something. I think you don't use ENIGMA at all, which is one reason for your lack of motivation.

So I don't think GsoC will help here either. We need people "in the know" who can fix long standing issues. We don't need features per se.

[Josh @ Dreamland]

Yes, the IDE is my number one pain point. I know Java; I have to work with it far more often than I like, actually. The issue is that even if I do add the resources and features I want LateralGM, the amount of work required to make it work well with the EGM format seems out of my reach. That's why I've started putting my chips on NaturalGM, which I think might warrant a spot in GSoC, if you pardon its small age. But ENIGMA? Yeah, I might let this one go....

I finished my MEI generator and am moving back to the flags header. A chunk of today will be spent doing my laundry (about three weeks' worth) and going grocery shopping. I might finish Flags today, but I doubt it. I'll probably finish it tomorrow after I get home from work. After that, I have one more pet project that is more interesting to me than ENIGMA, and then I guess I'll try to muddle through this merge once more.

I'll be a little more clear; the parts of ENIGMA that are now interesting to me are the parts of ENIGMA the IDE doesn't support. And I don't want to do Java IDE work. I don't really want to do IDE work at all, actually; Java isn't all that bad for it. But Swing is broken and LGM needs major surgery on its other two tiers for this, so what's the point?

[TheExDeus]

Another thing that was discussed at one point was some sort of implementation of "execute_string". Long time ago we people wanted to use V8 to somehow execute those scripts or use some version of clang with JIT. I'm looking at scripting languages now, but of course the best option would be to use EDL at runtime.

[daz]

I think if the community could come together and agree on an IDE to work on, that would be fantastic. As much as I think the idea of LGM is great, we need a better solution. Java isn't too bad as a language in my opinion, but it is just damn slow. I have 6 idle cores overclocked to 4ghz and an SSD, and it still takes LGM 2 seconds to open an interface window for the first time. Good lord. I am definitely willing to contribute to an IDE project (as I have done for LGM in the past), however I am not a) a great programmer, b) good at program design. I'm best at working on smaller details and functionality once the majority of it is up and working. So, we would definitely need a body or two knowledgeable with program design and ENIGMA -> IDE communications to start up a good bit of the work.

Anyway. In regards to my list: as I said, some of those are 2+ years old. I have not in fact used ENIGMA for over a year, I tried to start a project and switched to GM instead because the views system was working inconsistently. So I will have to go through my list and see if anything's fixed, broken, etc.

I would really love to see ENIGMA thrive.

Also in regards to the EDC, I would be willing to look at that as well, however last I checked there were really no instructions in regards to setting up a development environment for it. So if Josh or someone else would be willing to setup a guide for that and had a wishlist and buglist available, I'll dive right in. Last I checked, it was not a simple "install extension and get going", however if it works now out of the box without any other changes necessary then great I don't need instructions. Plus you know, you never even mention on the EDC's github page what version of SMF is minimum, and whether it's designed for 1.x only 2.x only, both, whatever.

At any rate, I do have more free time now than I've had in the past, but the majority of ENIGMA's missing features are way too complex for a newbie, and at this point I don't want to invest any more time into LGM.