ENIGMA Forums

Contributing to ENIGMA => Proposals => Topic started by: TheExDeus on November 23, 2014, 06:06:05 am

Title: Allow me to use C++ in EDL
Post by: TheExDeus on November 23, 2014, 06:06:05 am
I kind of hate GM datastructures. They are slower, harder to use and harder for compiler to optimize, because they are a wrapper. I would want to use STL containers, like vectors and lists. Can we finally make it happen? It's probably a parser thing again, but could it be possible in the short term to just add a language syntax to allow C++, like "{C++} { }". I know Josh already though about something like it, but I want to know how complicated would it be? Basically I want everything inside {C++} to NOT be parsed and taken as valid C++. Then every variable inside that scope to be temporary to the script (it should happen by default), so I can access it. Like for example:
Code: (c++) [Select]
{C++}{
#include <vector>
using std::vector;

class myClass{
 double value;
};

vector<myClass> myVector;
for (auto &cl : myVector){
  cl.value = 3.1415;
}
 
}

//This is parsed EDL
for (unsigned int i=0; i<{C++}{myVector.size();} ++i){
  ds_grid_set(grid, i, 0, {C++}{myVector.value});
}

This is very ugly, but the point is that I plan to make a very high performing part in pure C++ and then only copy the results back to EDL for drawing.
This way we don't need to wait for a new parser for 10 years, but get a "patch" that could potentially be easy to implement.
Any ideas?
Title: Re: Allow me to use C++ in EDL
Post by: sorlok_reaves on November 23, 2014, 10:23:23 am
I don't have an exact answer on this, but I have put some thought into it in the past. My thought process went like this:

Despite the challenges, I really like the idea of mixing EDL and C++, because it allows us (me) to easily experiment with different parsers, or different game engine scripting languages.
Title: Re: Allow me to use C++ in EDL
Post by: TheExDeus on November 23, 2014, 12:51:56 pm
Yes, headers are the biggest problems. But you can put classes inside the .cpp file as well and it's valid. You wouldn't be able to use templates though (they are purely a header thing). My idea was that this is a script. So everything goes in one script resource, which I then call. The problem of not parsing a whole "file" is that you then can't actually access neither local or global variables (for which parser generates accessors). So you can't actually do anything in that code that can be used elsewhere in the program. That is why I suggest some syntax, that allows me to select which part is parsed and which is ignored. So I can run fast code in C++, then give result back to an instance via EDL.
Title: Re: Allow me to use C++ in EDL
Post by: Josh @ Dreamland on November 23, 2014, 01:35:56 pm
We used to do what you're suggesting, if you remember. There's no syntax checking that way. I can't even tell you if you've dropped a semicolon. But even then, including vector wasn't an option, because you're still writing code in an event function.

As far as "Making a single 'file' parse as C++," we still do that today! It's called "definitions," and you can modify it in the ENIGMA settings.  If you want a hot fix, how about helping me out and petitioning for a formal, instantiable definitions resource like I've been bugging Ism (and anyone else who enjoys doing Swing work) for since 2008? I have every intention of allowing raw C++ definitions by that mechanism even when EDL is actually a capable language. We'll also want EDL definitions, so  you have a place to put global EDL functions (kind of like a file full of static scripts).

So if the parser takes ten years, fine; do this now, have it just add C++ definitions, and when the parser's done, that's one less fucking thing I have to worry about.
Title: Re: Allow me to use C++ in EDL
Post by: TheExDeus on November 23, 2014, 03:16:30 pm
But if I put the code in C++ definitions, then how can I access local or global variables for instances? Like how exactly can I use C++ definitions together with EDL? I guess I can use ds_ functions in the definitions, so I can just make a function that returns a data structure. That is the only thing that comes to mind.

Quote
But even then, including vector wasn't an option, because you're still writing code in an event function.
You mean we can't use vectors is because #include would end up in the function? If so, then headers and "using namespace" stuff can be put in the current C++ definitions file. When that is done, then I don't see why the vector wouldn't work inside a script.
Title: Re: Allow me to use C++ in EDL
Post by: Josh @ Dreamland on November 23, 2014, 09:54:55 pm
I'm sorry, I couldn't hear you over how great the code in write_object_data.cpp is.

(https://dl.dropboxusercontent.com/u/1052740/robert.gif)
Title: Re: Allow me to use C++ in EDL
Post by: DarkAceZ on November 23, 2014, 09:59:50 pm
I'm sorry, I couldn't hear you over how great the code in write_object_data.cpp is.
How can you compete with that, Josh? Why are you still even here? (I bet it's to suggest new shitty site layouts)
Title: Re: Allow me to use C++ in EDL
Post by: TheExDeus on November 24, 2014, 06:20:15 am
Josh wrote at least half of the code. Probably more.
Title: Re: Allow me to use C++ in EDL
Post by: Goombert on November 24, 2014, 02:55:00 pm
Quote from: Dreamfag
I'm sorry, I couldn't hear you over how great the code in write_object_data.cpp is.
Quote from: Babybitch
How can you compete with that, Josh? Why are you still even here? (I bet it's to suggest new shitty site layouts)

In all fairness, mine at least works and does inheritance, Josh's doesn't even work yet.
https://github.com/enigma-dev/enigma-dev/pull/825
Title: Re: Allow me to use C++ in EDL
Post by: Darkstar2 on December 01, 2014, 05:12:40 pm
Very much agree with you Harri in fact I have a few posts here I made a while ago about some stuff I wanted to do but was told I could not because of the stl containers and certain things not possible in EDL.....  I love your idea to embed native C++ in EDL, bypassing parsing.
There are good uses to this for certain application/games where GM structure is just inefficient.....mind you it's probably efficient for most people using GM lol :P  but there are certain things I want to do that push the limits, and require optimal speed in file and data handling......and short of writing your own C++ function and integrating it into the ENIGMA engine, this would allow proprietary C++ code outside the engine....


Title: Re: Allow me to use C++ in EDL
Post by: TheExDeus on December 01, 2014, 06:13:13 pm
Clearly this feature is way to complicated for the current parser. There is also a possibility this might not work for some reason (something Josh spoke about, but was not willing to elaborate).
Title: Re: Allow me to use C++ in EDL
Post by: Josh @ Dreamland on December 02, 2014, 12:35:15 am
You'd have to access those objects the same way you access them in other C++ functions. Ie, using instance_event_iterator.

The main reason you can't use vector with the current compiler is that it's terrible at coercing types. It was always bad at it, but it's actually gotten worse over time. Originally, you could access vector like an array, but couldn't call functions because the type resolution wasn't good enough to get accurate function information from template types, so I never bothered coding checks for member function calls. I fixed the template types issue in JDI, but in the meantime, someone broke it to the point that it doesn't even realize anything in the code is not a var, anymore. No idea why. Don't feel like investigating.
Title: Re: Allow me to use C++ in EDL
Post by: TheExDeus on December 02, 2014, 04:21:32 am
Quote
You'd have to access those objects the same way you access them in other C++ functions. Ie, using instance_event_iterator.
That is why I proposed an inline operator which just ignores parsing. So if I need to access an instance variable, I allow the parser to generate it, like so:
Code: [Select]
ds_grid_set(grid, i, 0, {C++}{myVector.value});So grid will turn into the correct accessor, while myVector.value will not be parsed and be valid C++.
Title: Re: Allow me to use C++ in EDL
Post by: sorlok_reaves on December 02, 2014, 08:21:40 am
I think you'd also need:

Code: [Select]
{C++_instance}{std::vector<int>myVector;}
...where "C++_instance" means "put this in the object declaration for this object type. But what if you declare an integer? Then you'd need:

Code: [Select]
{C++_ctor}{my_var = 10;}
...where "C++_ctor" means "put this in the object's constructor". If you allocate memory, there's a strong argument for "C++ dtor" as well. Oh, and possibly "C++_header", which goes at the top of IDE_EDIT_objectdeclaration.h.

This is one way to do it "without any parsing". We could also parse the code and extract #include<> statements, removing the need for "C++_header"  ...but now we've gotten into parsing C++.

I think this is why everyone is hesitating on the issue ---there's no easy way to hack it in, and (for now) there's no way that's clearly the "right way" to do it.
Title: Re: Allow me to use C++ in EDL
Post by: TheExDeus on December 02, 2014, 10:15:39 am
Quote
...where "C++_instance" means "put this in the object declaration for this object type. But what if you declare an integer? Then you'd need:
That is if I want to use those classes or types in several code snippets. I only required the vector<> to be defined inside one function (e.g. a script) and it can be deleted when the function ends. I don't need to declare in create and use in step. So in my case only {C++} would probably work just fine. So the whole idea was that in performance critical script, I can use pure C++, then only copy back the results for local instances. Look at my original code snippet, the myVector is not used outside of that code. It gets deleted when when the script ends. That is why I copy the values back into ds_grid.

If you need something more longer lasting you could use a global variable, but I guess globals are also defined in some declaration and they are not just globally scoped.