ENIGMA Forums

General fluff => Announcements => Topic started by: Josh @ Dreamland on February 07, 2015, 11:56:23 AM

Title: Google Summer of Code
Post by: Josh @ Dreamland on February 07, 2015, 11:56:23 AM
Hello, everyone; long time, no see.

Google Summer of Code signups are around the corner. Are we, as a group, interested in participating? I am considering sponsoring either ENIGMA or NaturalGM, or even both, depending on which project is able to generate an workable list of tasks for contributors.

The problem with ENIGMA's participation is that its contributor base is already mostly college-level help, so the majority of tasks that would be easy to delegate to new help are already completed, while the remaining work requires more skill and background info on what ENIGMA is and what it is supposed to be.

That said, looking over the wiki page from last year (http://enigma-dev.org/docs/Wiki/Google_Summer_of_Code:_2014), some of the workpoints seem nearly feasible. The problem is that no one has started any of them. If we want help, we have to be serious about having room for participants to actually contribute code. We can't be allocated a new contributor and expect them to figure out what ENIGMA is and just start tacking entire new systems on. Work points such as "Implementation of an Asynchronous Networking System" are entirely too much work for someone who is unfamiliar with the project. Does the Network_Systems folder even work like the other systems folders? How will a new contributor even know they are on the right track?

If we are to participate, we need tasks that begin with ten lines of code and end with implementing an entire system. There needs to be room for a contributor to start small and grow in the project. I can't endorse ENIGMA with such course-grain tasks. This is the reason I stayed out of the GSoC planning last year, and if we can't generate a better task list this year, we'll hit the same fate.

We can improve the "Implementation of an Asynchronous Networking System" task by creating a reference list of what we actually expect this system to look like, and offering an example game which, given these functions, should be able to be played over a network. We can't expect a new contributor to write the game that tests his or her new functions. That's way too much work for someone who has never heard of the project.

Related: Do we even have a simple dev wishlist, anymore? I occasionally hear about X huge thing that is completely broken and everyone has been working around, but I have essentially no idea what people want to see out of this project right now, aside from a new compiler. It seems to me that we have no little tasks; the project is waiting on someone to "just go through and fix everything." Does anyone care to prove me wrong?

To be brief, if we want to participate in GSoC, we have to make sure we have tasks that are actually suited to someone who has never heard of the project. Otherwise, I guess we can sit this one out.
Title: Re: Google Summer of Code
Post by: Josh @ Dreamland on February 07, 2015, 12:02:31 PM
The rebuttal I'm receiving is that it is the job of the mentor to provide guidance for the recruits. This is certainly the case, but we can't postpone things like authoring test games until the student has already created code. We should have a sample game or other test suite to give the student as soon as they join. At very least, in the case of a networking system, we should have a game that is working except for the networking bits, which can be broken, commented, or completely missing—as long as there is minimal effort to adding them in. We can't have our mentors scrambling to write a buggy test game while the student hangs around waiting. That would be a waste of the student's time and Google's money.

And yes, students are suppose to be somewhat autonomous and generate their own work items—this does not mean that we can leave the mentor and student to completely improvise the entire process. Some real planning ahead of time is still required. At very least, students should have a breadcrumb trail to follow in the absence of the mentor.

Here is an interesting news bulletin: I am allowed to mentor as a Google employee, and current contributors who are also students are allowed to sign up.
Title: Re: Google Summer of Code
Post by: daz on February 07, 2015, 03:30:27 PM
Frankly, I don't think anyone else around here is knowledgeable enough to be a mentor other than you, Josh. Hell even the last time I tried to do something big to Enigma (Android), that was 99% you. And I just had to give up, because there was no starting point for "hey yeah this is how these systems work together and connect to the engine core", plus my C++ is hilariously weak to begin with [and when you weren't around, which was most of the time I just couldn't progress which made it too frustrating to continue]. Hell all of that work is now lost to the ether after I moved to an SSD, the only remnants are some vague instructions I left on the wiki which assumes the user has a copy of an Android fork. Maybe I should have used Github for some poor unfortunate soul of the future. I'd say there needs to be a lot of documentation done if you want someone to ever get up to the system or platform level. This project is a relatively undocumented mess that is pretty hard to jump into. And yes, LGM is a huge pain point as well. I'd be in favor of putting another IDE for GSoC instead, because I would like that option.

About networking: last I tried, Berkley sockets worked. What networking are you even looking to get done?

As for feature requests and what not, those are a dime a dozen. Enigma is missing an assload from GM still for starters.
Title: Re: Google Summer of Code
Post by: DaSpirit on February 07, 2015, 03:46:41 PM
Tasks such as "fix this" or "add these few functions" are not good tasks. We need to think of large tasks that would take three months to finish for someone who is completely new to intermediate to the topic.
Title: Re: Google Summer of Code
Post by: Josh @ Dreamland on February 07, 2015, 04:07:33 PM
Dime a dozen tasks are great. They're a great way to get a new contributor interested in the system. I think we'll hit a snag trying to sell some of the larger tasks to people. Keep in mind, the candidates are going to be paid, but they still get a say in the project they work on. We're competing with how interesting a task we can generate.

So, can anyone list any of these "dime a dozen" tasks?
Title: Re: Google Summer of Code
Post by: daz on February 07, 2015, 04:27:32 PM
I'm going to just list a bunch of random things that might have already been completed or might be an insane amount of work. I have had a wishlist, and some of these might be up to 2 years old.


That would be a good start anyway.

edit--
Someone would need to go through the code and check what functions have been implemented and what haven't, from GM. Some of those might be easy to implement, but the wiki is often not updated when a function is implemented, or whether it is still a stub or completely missing.
Title: Re: Google Summer of Code
Post by: TheExDeus on February 07, 2015, 04:35:43 PM
Quote
Proper way to export games instead of having to go to temp files and copying the executable
This has actually been fixed.

I only need a new parser/compiler. I don't know what else people can do, besides implement a million features. I agree that ENIGMA is very undocumented and can be hard at times, but hell, a person made a LUA extension in one day (at least compiling), so we haven't totally failed. I think we need more documentation and a new website with tutorials and stuff. We should probably go trough GitHub issue pages and fix bugs. I have actually fixed many of them, but as nobody else tested for them, the issue remained open. So I guess part of those issues are already resolved.

I guess another thing a person can do is Android support of some sort. Like daz said, the first step is getting a project to compile into apk, then the person could change either the current GL3 rendering system or create a GLES2 one, that would allow porting games.
Title: Re: Google Summer of Code
Post by: Josh @ Dreamland on February 07, 2015, 05:03:53 PM
dazappa:

Harri: are you currently a student? Or, are you interested in mentoring?
Title: Re: Google Summer of Code
Post by: Josh @ Dreamland on February 07, 2015, 05:11:03 PM
I have created a new Wiki page (http://enigma-dev.org/docs/Wiki/Google_Summer_of_Code:_2015) for these tasks. Feel free to edit the page to elaborate on tasks. If you add a task, please also begin a discussion about it here. Please don't delete a task without discussion beforehand.
Title: Re: Google Summer of Code
Post by: TheExDeus on February 07, 2015, 06:43:30 PM
Quote
Sound system improvements. I don't know if the new audio engine from GM has been implemented or not, but last I checked even sound_volume did not work for the basic system. Sound position.
I think most of them worked. What is missing is sound effects, but I don't think we can really add them to existing systems. We also have both OpenAL and DirectSound which seems redundant, as one is not really better than the other.
Quote
Particle systems with advanced features: attractors etc
Looking at the source it seems deflectors and attractors are in. I think the system was finished. The problems was with rendering, because for every graphics system a bridge was needed. As as quick fix I added PS_particle_bridge_fallback which uses General drawing functions to draw particles. This is slower than having a special rendering function, but it at least works on every graphics system. A GsoC task could be making the particle system faster, better or even implementing it on GPU vai OpenCL or something. It's "hard", but 100% completable in 3 months. Especially if nothing really has to be changed ENIGMA side and can be done by modifying the extension.
Quote
Views system is extremely inconsistent when compared to GM
Elaborate, because I fixed view_angle which was the last known bug I was aware of. There can be some more inconsistencies, but I use views all the time and haven't noticed any problems. I don't use GM though, so I might not know about something. One thing we are missing we potentially need is view_surface, which allows easier post-processing effects.
Quote
Pixel perfect collisions
That should work now.
Quote
Polygonal collisions with collision editor
Would love that.
Quote
Full box 2d integration, complete with editors and other IDE integration
Box2D should be working fine as functions, but it's not implemented in the IDE. So that can be done.
Quote
Working HTML5 export via emscripten or equivalent We have this. It was never hooked into LateralGM because it and the compiler are not extensible enough.
How "working" was it? Remember seeing it, and it had the basics right, but that didn't involve making an HTML5 project via LGM. It also involved coding JS engine from scratch, which can potentially be bypassed using emscripten.
Quote
Game debugging.
Robert started working on that and I would love this too. I actually started a discussion on how this would work, because we need to map the user code to parsed C++ code, which didn't seem totally trivial.
Quote
Game profiling
I started on a primitive GPU profiler, but it wasn't meant to be anything fancy, so it would probably not fit. It would just return how many draw calls are done, how many vertices are drawn, how many models and textures are used and so on. But no timing or GPU temperature or something like that.
Quote
Texture groups support like GM
This is actually needed. Texture groups are called "texture atlases" or can be implemented with "texture arrays" or in more recent OpenGL "sparse texture arrays". It requires a slight recode of the texture system and modifications to all drawing functions. I want this because it's most useful for 2D games and applications (which I mostly make and test). Like if I could put the font texture in the same texture page as my sprite, then draw calls would plummet in my newest project from 500 to 1.

Quote
Harri: are you currently a student? Or, are you interested in mentoring?
Currently I'm a Phd student. I'm not sure about mentoring though, as I cannot promise anything time wise.

What are you currently doing? It seems you are very time consumed with your job, as I haven't seen any commits by you in any of the ENIGMA or ENIGMA related projects.

An IDE suggestion would be custom resource support. Allowing everything to be saved in EGM (it's a zip file, it should already support everything with 100% backwards compatibility, but for some reason it's broken now). So it would be possible to add an extension, that would add a resource or even an ide to view it or edit that resource (which can be done now, as when you open sprite in your own paint program). Like I'm making a GUI system for ENIGMA. I would want a gui editor in LGM, but adding that would be painful for me. So I can make an external editor (in ENIGMA itself), that can open my hypothetical GUI save file and edit it. Then save it and LGM reloads it. And then I can use it in code. Same with shader editor. Making a shader editor in LGM is practically impossible. But making one ENIGMA isn't.
Title: Re: Google Summer of Code
Post by: Josh @ Dreamland on February 07, 2015, 08:44:23 PM
I'm afraid it's not that simple. It isn't that I have so little free time—it's just that when I sit down to fix one of ENIGMA's problems, I find myself traversing a whole loop of increasingly large problems until I lose interest and stop working. There are so many broken things with it that I just don't want to fix any of them, because I have this feeling that it makes no difference in the long run. And the amount of time required to fix one item correctly isn't itself all that high—it's just that I only have two days a week that are all mine, and it isn't long enough to fix something completely. And when I start to fix something and am then interrupted by a week of work, I lose interest in it, and then it gets bit rot. So really, it's more an issue with contiguous ENIGMA time. Even when I do take a week off, I spend it visiting family. So...

I actually have more free time now than I did in college. The problem is focusing on ENIGMA long enough to make a difference. When I was young, I'd make a plan and stick with it even if evidence (read: Rusky) suggested it would eventually fail. This is why anything ever got done. Now that I'm older and can see around corners, it's not so easy to just continue down the path of gradual improvement.

Most of the little things in ENIGMA are done. The project right now probably doesn't really need people to do little things so much as it needs someone to finally get the time to do big things. But I figured it'd be worth bringing up GSoC anyway. The reason I created this thread is because I'm not personally convinced we should participate in GSoC, for just those reasons. But if you can convince me, I can convince Google.

If you want to know what I'm working on this moment, it's a program that reads in a video and produces MEIs of frame intervals of a given length. The reason for this is complicated and involves my landlord. Before that I was working on a new flags library, which is for general-purpose use. Before that I wrote a world editor for Terraria worlds. Before that was Christmas, and around that time, I recoded a piece of ENIGMA's compiler, and eventually did merge that in. So that's one less thing that needs done before I can actually get anywhere with it. Before that, I don't really remember.

I don't know that there's a simple answer to what it will take for me to sit down and finish that new compiler. IDE support for all the wonderful things dazappa is requesting would certainly help. Maybe I should nip this in the bud and just write a damn IDE. If I finish the widget framework I started in ENIGMA ages ago, that would probably help. But even then, ENIGMA on Windows is a nightmare. I don't know how you put up with it. I guess you're clever enough to set up your own compiler so you don't have to put up with the release zip cheeseboy started Robert compiling. The whole thing is mucky, and I just don't like touching it. And every little administrative task wastes so much goddamn time. I spent four hours today trying to figure out how to set up an SSL certificate for ENIGMA's site. I've wasted tons of time looking for a good license. The EDC's a wreck; I've wasted hours on it, too. There are things I'm not good at, which I normally end up doing, too. It apparently doesn't need done that bad, of course, because my attempts to just ignore it all have been largely successful. But then it just balls up and I don't want to go back.

I mean, if all you want is a new parser, it's in the branch. It doesn't support all the things I wanted it to, but it's correct. It will want the new JDI, which is in another branch. I went to merge those, but my lexer changes were incompatible, so I went to consolidate those so future changes would be easier to merge in. I then discovered a couple problems with the way JDI handles macros, and went to fix those. It got messy and resulted in a slowdown, so I elected to start replacing the macro system, which I hoped would solve both problems. I got distracted doing other things, including the fucking EDC and license problem, and never finished.

I imagine that gives you some idea of how I feel.
Title: Re: Google Summer of Code
Post by: TheExDeus on February 08, 2015, 06:59:05 AM
I get that feeling. But from functional standpoint those are the only two issues I have with ENIGMA - a parser that doesn't parse valid EDL/GML and so I require workarounds, and LGM which often crashes and corrupts EGM's. And what bugs me is that both are out of my personal expertise. I can make many things in ENIGMA engine, but I sadly cannot do anything of that sort. While I'm working on my projects for next few months I will probably implement a lot more things in the GUI extension, fix up many renderings things I will encounter, add an extension for scripting language (either LUA or AngelScript) and more. And I will do that because ENIGMA is the IDE part of my project - I make a computer vision program that allows you to perform CV with nodes and real-time scripting. Everything behind the scenes will be an C++ extension for ENIGMA which can be standalone compiled. While the visual part is coded in ENIGMA. This makes me work on ENIGMA. Maybe you also need to take a project which can be done in ENIGMA and do it there? Like your Terraria editor (which I guess wasn't for Google) - It could of been coded in ENIGMA, and while making it you would probably fix some bugs or implement something. I think you don't use ENIGMA at all, which is one reason for your lack of motivation.

So I don't think GsoC will help here either. We need people "in the know" who can fix long standing issues. We don't need features per se.
Title: Re: Google Summer of Code
Post by: Josh @ Dreamland on February 08, 2015, 08:28:01 AM
Yes, the IDE is my number one pain point. I know Java; I have to work with it far more often than I like, actually. The issue is that even if I do add the resources and features I want LateralGM, the amount of work required to make it work well with the EGM format seems out of my reach. That's why I've started putting my chips on NaturalGM, which I think might warrant a spot in GSoC, if you pardon its small age. But ENIGMA? Yeah, I might let this one go....

I finished my MEI generator and am moving back to the flags header. A chunk of today will be spent doing my laundry (about three weeks' worth) and going grocery shopping. I might finish Flags today, but I doubt it. I'll probably finish it tomorrow after I get home from work. After that, I have one more pet project that is more interesting to me than ENIGMA, and then I guess I'll try to muddle through this merge once more.

I'll be a little more clear; the parts of ENIGMA that are now interesting to me are the parts of ENIGMA the IDE doesn't support. And I don't want to do Java IDE work. I don't really want to do IDE work at all, actually; Java isn't all that bad for it. But Swing is broken and LGM needs major surgery on its other two tiers for this, so what's the point?
Title: Re: Google Summer of Code
Post by: TheExDeus on February 08, 2015, 02:21:59 PM
Another thing that was discussed at one point was some sort of implementation of "execute_string". Long time ago we people wanted to use V8 to somehow execute those scripts or use some version of clang with JIT. I'm looking at scripting languages now, but of course the best option would be to use EDL at runtime.
Title: Re: Google Summer of Code
Post by: daz on February 09, 2015, 04:02:45 PM
I think if the community could come together and agree on an IDE to work on, that would be fantastic. As much as I think the idea of LGM is great, we need a better solution. Java isn't too bad as a language in my opinion, but it is just damn slow. I have 6 idle cores overclocked to 4ghz and an SSD, and it still takes LGM 2 seconds to open an interface window for the first time. Good lord. I am definitely willing to contribute to an IDE project (as I have done for LGM in the past), however I am not a) a great programmer, b) good at program design. I'm best at working on smaller details and functionality once the majority of it is up and working. So, we would definitely need a body or two knowledgeable with program design and ENIGMA -> IDE communications to start up a good bit of the work.

Anyway. In regards to my list: as I said, some of those are 2+ years old. I have not in fact used ENIGMA for over a year, I tried to start a project and switched to GM instead because the views system was working inconsistently. So I will have to go through my list and see if anything's fixed, broken, etc.

I would really love to see ENIGMA thrive.

Also in regards to the EDC, I would be willing to look at that as well, however last I checked there were really no instructions in regards to setting up a development environment for it. So if Josh or someone else would be willing to setup a guide for that and had a wishlist and buglist available, I'll dive right in. Last I checked, it was not a simple "install extension and get going", however if it works now out of the box without any other changes necessary then great I don't need instructions. Plus you know, you never even mention on the EDC's github page what version of SMF is minimum, and whether it's designed for 1.x only 2.x only, both, whatever.

At any rate, I do have more free time now than I've had in the past, but the majority of ENIGMA's missing features are way too complex for a newbie, and at this point I don't want to invest any more time into LGM.
Title: Re: Google Summer of Code
Post by: DaSpirit on February 10, 2015, 10:28:36 AM
That IDE would probably be mine. Josh has been talking about Natural GM for quite a while. When I originally started making it, I've been constantly asking him for his opinion on my design.

So where am I now? Well, I restarted it on January, mostly because I was changing it entirely - the biggest change being the introduction of automatic memory management for plugins. Plugins will automatically clear their memory when they are unloaded and the IDE itself will have no problem when their memory is destroyed because the IDE stores memory as weak references (except for the systems where the plugin manager is closely tied to, where I can delete the memory easily). I have most of it figured out, and was hoping to work on it during Google Summer of Code. Plugin loading is currently functional (with the exception of the dependency system) and plugins will be able to create menu/toolbar items soon. Afterwords, I will be working on the project tree and editor API (which I will simply be cutting out from my old NGM version).

I am very pleased to hear that people want an IDE. My main repository is currently private on BitBucket, but I can put it on Github if people are willing on contributing to it. Robert has contributed it a few times already actually and I have a list of small tasks that he's been implementing. So yeah, are people willing to contribute by implementing small tasks while I work on the API?
Title: Re: Google Summer of Code
Post by: TheExDeus on February 10, 2015, 01:34:39 PM
I think people would be very interested in seeing NGM going forward. I haven't looked at the code yet, so I'm not sure what I can contribute, but as it's C++ then it's certainly more than I could for LGM.
Title: Re: Google Summer of Code
Post by: polygone on February 15, 2015, 05:30:13 AM
http://enigma-dev.org/docs/Wiki/ENIGMA:Todo

Seems to me though that the problems and bugs seem to be piling up, there's some years old now. So instead of adding more and more additional systems and producing more bugs in those so I would advice halting further functional progress and going through a thorough extermination process, vastly improving ENIGMA's error reporting and DEBUG functionality and finally getting around to setting up a build bot system that checks for regressions in new commits.
Title: Re: Google Summer of Code
Post by: daz on February 18, 2015, 04:25:44 PM
Just something else I discovered today while toying with the idea of working on the EDC: it uses Rackspace cloud for storing the files and screenshots for added games. Great, except that to attempt to develop any of that code further (e.g. multiple download link support), I'd need to sign up for Rackspace cloud. And their plans start at $50/mo...

DaSpirit there is no harm in putting it on GitHub. I wasn't even aware that anyone was still working on IDE projects any more.

Also Josh, I'm sure you have a good bit of money now, why not hire a lawyer to create a custom license for ENIGMA?
Title: Re: Google Summer of Code
Post by: TheExDeus on February 18, 2015, 04:50:46 PM
Can't use the EDC Rackspace account for developing?
Title: Re: Google Summer of Code
Post by: DaSpirit on February 18, 2015, 07:37:25 PM
As the final date for GSoC organizations to apply is approaching, I should say that we decided not to participate in GSoC this year. It was difficult to set up the current ideas list, which is the sign to us that it is not time. Josh hopes that we can participate next year, when ENIGMA advances more and hopefully, we have a new IDE. I hope that Josh finishes his current 5 projects that are unrelated to ENIGMA by next year and finally fixes everything. :3

DaSpirit there is no harm in putting it on GitHub. I wasn't even aware that anyone was still working on IDE projects any more.
Yeah, will do soon. I recently got swapped with school work, so might do at the end of next week. I hope you guys want to participate. I have so many warnings from everything that I didn't implement yet because I didn't deem it to be too important at the moment that are wrecking my OCD. :p
Title: Re: Google Summer of Code
Post by: Josh @ Dreamland on February 18, 2015, 09:08:01 PM
daz: I have paid literally seventeen cents for my Rackspace Cloud Files so far. Cloud files is not Cloud Servers. They charge by the megabyte, and we're using a couple dozen.

And yes, our GSoC participation is out the window due to lack of proper coordination, and for an overarching reason: more minimum-time contributors is not what we need right now.