General fluff => General ENIGMA => Topic started by: Josh @ Dreamland on May 26, 2013, 10:48:36 am

Title: Website and IRC Problems Version Control Could Fix
Post by: Josh @ Dreamland on May 26, 2013, 10:48:36 am
Sorry to add yet another topic to this already overpopulated board. Maybe one day the whole forum will be this active, and I won't have to feel bad. I figure this is a good board to pollute, since people are already paying extra attention to it.

To cut to the chase, we need to start using version control EnigmaBot and other features of the website.
I have organized my points into •B•U•L•L•E•T•E•D• •L•I•S•T•S• to save everyone some brainpower and reading.


The idea: EnigmaBot and the SMF/Wiki extensions we have written need placed under version control, possibly in the same repository, at some point down the road. 

Problems we need to address with which this change may or may not help:
Writing this has given me carpal tunnel.

Ism and I will get everything moved into a git repository and checked for sensitive information/passwords/etc at some point in the discrete future following discussion here. Any additional problem you have with the above, list them, as they are likely to be dealt with long before the site is on GitHub.
Title: Re: Website and IRC Problems Version Control Could Fix
Post by: polygone on May 26, 2013, 11:48:54 am
I want to be able to link to 64digit games on the EDC.
Title: Re: Website and IRC Problems Version Control Could Fix
Post by: Josh @ Dreamland on May 26, 2013, 01:11:57 pm
Title: Re: Website and IRC Problems Version Control Could Fix
Post by: forthevin on May 26, 2013, 03:27:34 pm
I think this sounds like an excellent idea, and the plans seem sound overall.


In regards to the license, version 2.0+ of SMF uses a 3-clause BSD license, which is compatible with GPL3. As far as I can tell from the description on Wikipedia as well as the license description (http://www.simplemachines.org/about/smf/license.php), redistribution of source code or software license with the Simple Machines License, as well as any modifications, requires written permission from SMF. The themes from SMF 1.x are licensed under the Simple Machines License, but I don't think it would make any sense if the webpages generated using a theme are also licensed under the Simple Machines License, since then even basic usage of SMF would require written permission. Therefore, given that we are not currently, and do not plan to, redistribute either SMF or any themes for any version, or any modifications of those, I am convinced that there are no issues.
That said, in the long-term it would be nice if we could upgrade the theme to 2.0+, but I am convinced that there are no problems with the current plans.


Excellent plans. I have personally thought about adding automatically run tests to ENIGMA, primarily as an aid to development. I also think some of those tests could be run for releases, to see statistics of the tests (how many passed, how many failed, etc.), and possibly reject releases that do not work at all. A tests page could also be added to the site, and tests run daily/weekly or something like it, and statistics shown for each run, such that developers could more easily track regressions in bugs and see which bugs have been solved.


Good points in general.

In regards to the download links, it would be useful to have platforms for each download link. I also think it would be useful to have some sort of option for playing the game directly on the page or on another page instead of downloading, when we get HTML5 support. That could be very nice. An option for downloading the source would also be very nice.

I think it will definitely be necessary to be able to search for games. I also think adding tags to a game, such that users can search for names and/or certain tags (such as genres), is a good idea.

An updating/reupload mechanism for games may also be a good idea, in case the user adds more content or fixes bugs.

In regards to viruses, there needs to be a mechanism at some point for "moderators" to remove games that contain/are viruses. I don't think it is needed in the start, but as the EDC grows, I think it will be necessary in order to scale.

And polygone is being silly, Josh is confused, and Robert says he is happy today.
Title: Re: Website and IRC Problems Version Control Could Fix
Post by: IsmAvatar on May 26, 2013, 03:35:11 pm
While I certainly don't oppose the idea, I do, however, feel like this is just another case of "stick it on github and the problem is magically solved". The problem with this ideology is twofold: 1) Interested manpower. Most of our interested manpower is already working on the wiki and enigma, which have higher priority than the bot and the website (as they should). As for generating interest, that leads to point 2) Generalizability. The bot and the website are pretty much singletons. They don't generalize to anything else, so there's not really a need for a second one to be launched anywhere else on the entire internet. Generally a project is best suited for source distribution and version control when it's expected that there will be multiple instances of it.

This leads to a more serious issue: more leechers than seeders. With so few people actually contributing to these projects, the only thing opening the source code effectively does is allow people to read it, learn from it, and abuse it. We're revealing our security flaws, but are we fixing them? If we're not, people will use those security flaws.

Just my two cents and concern. At any rate, I'd love to see them both worked on more, but I think we should hesitate a little before we just jump to "throw it on github for all to see".
Title: Re: Website and IRC Problems Version Control Could Fix
Post by: polygone on May 26, 2013, 03:51:02 pm
I want a way of knowing if somebody makes a comment on a game I posted at the EDC.
Title: Re: Website and IRC Problems Version Control Could Fix
Post by: Josh @ Dreamland on May 26, 2013, 10:58:01 pm
Polygone[0]: I guess I can see merit in linking 64Digits files, but then we don't have control over whether the user can delete the file without deleting the game. If the EDC controls the files, then users can't accidentally break links.


Regarding the licensing issue: those were basically my thoughts, though the attribution clause concerns me; GPL does not require attribution except insofar as the license remains intact, and whether that is sufficient to meet the three clauses of that BSD license was unclear to me. Also, that still presents an issue if we want to put the theme up on GitHub, though I don't see how they could limit our ability to do that if their license is GPL compatible.

Regarding the regression tests: I've wanted to make a regression testing suite for a long time, but I never had the wherewithal to get the job done myself. The task is every bit as daunting as documenting all the functions; it'd be an endeavor for the entire team. The most intricate testing suite we ever had was TGMG's compiler test, which, using a collection of a couple thousand examples harvested from 64Digits, compiled each one sequentially with ENIGMA, noting missing functions. I have no idea what has become of this testing environment, but TGMG should be back with us in a month or so.

Regarding your EDC points: I had meant to express that the multiple download links were for each platform, but I forgot that they might also be used for mirrors and assumed that the idea was implied. But yes, that feature is definitely on the radar. I really like the HTML5 embedder idea; I was actually thinking about using TGMG's old Java applet that embeds ENIGMA games (talking of GMBed). Then users could play any ENIGMA game from within their browser, sort of (the game would still be an EXE running on their machine, but it'd give the illusion of running in their browser). I do like the HTML5 idea, though; I forgot to consider it. You can currently edit your own EDC submissions, but the thing is presently devoid of moderation features. So if someone makes an ass of themselves on it, I have to delete them manually using SQL queries, which will probably piss me off enough to IP ban that person. That's definitely also on the radar. What 64Digits (the site upon which the EDC is based) does is use an approval queue for all games; games aren't displayed until a moderator checks that it isn't a virus. Of course, that only works if it's not on a timer, or something, but it's still one decent measure.

IsmAvatar: I hope you're wrong about this, for once. Most of my idea is that security isn't an issue. The reason for this is that all of the SQL queries go through the SMF API, which handles all sanitation for you. Thanks to that, the only kind of exploits anyone will find are generic JavaScript-flavored XSS. The database access is all handled by SMF's library, which would remain far from the GitHub. And yes, most of our manpower is tied up in the engine, and that is right where it should be. But I mentioned the case with Robert as a good example of something versioned site code will fix; now no one needs me to edit theme-related code, because it's all right on GitHub. Meanwhile, the stuff I do *not* want people editing, is *not* on GitHub. Granted, if Robert wastes more time on that, that's still time he could have been spending on the project. But he's going to waste my time, too, if that's not an option. And keep in mind, some people will come along who want to contribute, but don't know any C++. Some of those will be web developers who might want to lend a hand.

Polygone[1]: That can be arranged.
Two other missing features of the EDC:

1) It does not correctly set the currently active page in the forums. It shows up as "Unknown action."
2) There's no way to watch threads/blogs/users/general activity.
Title: Re: Website and IRC Problems Version Control Could Fix
Post by: forthevin on May 27, 2013, 02:58:30 pm
In regards to the licenses, I am certain that GPL3 and 3-clause BSD are compatible, since the Free Software Foundation states that they are compatible: http://www.gnu.org/licenses/license-list.html#GPLCompatibleLicenses.
In regards to distribution, the issue is that SMF versions 1.0 and 1.1 are licensed under the Simple Machines license, while SMF 2.0+ are licensed under 3-clause BSD. I looked into the source for SMF 1.1, and their themes are also licensed under the Simple Machines license. Therefore, the themes cannot be redistributed without their permission.
I am not certain whether we can redistribute the SMF 2.0+ software, though it may be allowed, since I don't think using SMF 2.0+ with a 1.x theme counts as a modification or derivative to the 1.x theme.

In regards to security, I think there's a better view of things: I think it is a safe bet that no matter what we do, the website is going to get cracked sooner or later. Open sourcing parts or not will not make much of a difference in regards to security, either positively or negatively, but it will make a considerable positive difference in regards to development and maintenance. Therefore, I think the best idea is to version control the things we want to version control, try our best to make things secure, and otherwise ensure that nothing is lost when the website gets cracked. This means that we shouldn't have any personal information anywhere, that we should discourage people from writing things they don't want in public in PMs, and encourage people to only use passwords on the website that are entirely unlike any of their other passwords, and finally backup anything that we would not like to get lost.
One of the things that convinced me of this was looking into how SMF hashes passwords. In the current version of SMF, SMF 2.0.4, password hasing is done by using the username as the salt, and using SHA-1 for the hashing. This is not a very good way of doing things, as explained on this website: http://crackstation.net/hashing-security.htm. Given that SM does not get that part right, I suspect that there are other parts in SMF that they do not get right either.
Title: Re: Website and IRC Problems Version Control Could Fix
Post by: IsmAvatar on May 28, 2013, 10:26:26 am
Sounds good to me. Just beware before you throw EnigmaBot up on the github, as I know for a fact that he stores his credentials in either a plaintext file or hardcoded, which gets "distributed" right along with him. This information would naturally need to be censored. Otherwise, I'd like to see him AND thundercleese (thevalog) up on version control so that we can begin merging them. Again, censoring where appropriate with thundercleese.
Title: Re: Website and IRC Problems Version Control Could Fix
Post by: Josh @ Dreamland on May 28, 2013, 04:40:56 pm
Thundercleese is already on GitHub, probably with a full SQL password along with Gary's IRC credentials. I'm aware of EnigmaBot's credentials file, and I'm weary of other files with SQL passwords.