Allow me to use C++ in EDL

[TheExDeus]

I kind of hate GM datastructures. They are slower, harder to use and harder for compiler to optimize, because they are a wrapper. I would want to use STL containers, like vectors and lists. Can we finally make it happen? It's probably a parser thing again, but could it be possible in the short term to just add a language syntax to allow C++, like "{C++} { }". I know Josh already though about something like it, but I want to know how complicated would it be? Basically I want everything inside {C++} to NOT be parsed and taken as valid C++. Then every variable inside that scope to be temporary to the script (it should happen by default), so I can access it. Like for example:

Code: (c++) [Select]

{C++}{
#include <vector>
using std::vector;

class myClass{
 double value;
};

vector<myClass> myVector;
for (auto &cl : myVector){
  cl.value = 3.1415;
}
 
}

//This is parsed EDL
for (unsigned int i=0; i<{C++}{myVector.size();} ++i){
  ds_grid_set(grid, i, 0, {C++}{myVector.value});
}
This is very ugly, but the point is that I plan to make a very high performing part in pure C++ and then only copy the results back to EDL for drawing.
This way we don't need to wait for a new parser for 10 years, but get a "patch" that could potentially be easy to implement.
Any ideas?

[sorlok_reaves]

I don't have an exact answer on this, but I have put some thought into it in the past. My thought process went like this:

• Making a single "file" parse as C++ is pretty easy. (By "file" I mean "event, script, or timeline moment"). We just turn the parser off for that file.
• A bracketed syntax like you propose is harder, since we need to integrate it into the parser.
• One of the main challenges of code like yours is organization. Where exactly would this go? In some object's create() method? It looks like the class definitions should go into their own "header-like" area, and the initialization and ds_grid_set() code would go into the actual code.

Despite the challenges, I really like the idea of mixing EDL and C++, because it allows us (me) to easily experiment with different parsers, or different game engine scripting languages.

[TheExDeus]

Yes, headers are the biggest problems. But you can put classes inside the .cpp file as well and it's valid. You wouldn't be able to use templates though (they are purely a header thing). My idea was that this is a script. So everything goes in one script resource, which I then call. The problem of not parsing a whole "file" is that you then can't actually access neither local or global variables (for which parser generates accessors). So you can't actually do anything in that code that can be used elsewhere in the program. That is why I suggest some syntax, that allows me to select which part is parsed and which is ignored. So I can run fast code in C++, then give result back to an instance via EDL.

[Josh @ Dreamland]

We used to do what you're suggesting, if you remember. There's no syntax checking that way. I can't even tell you if you've dropped a semicolon. But even then, including vector wasn't an option, because you're still writing code in an event function.

As far as "Making a single 'file' parse as C++," we still do that today! It's called "definitions," and you can modify it in the ENIGMA settings.  If you want a hot fix, how about helping me out and petitioning for a formal, instantiable definitions resource like I've been bugging Ism (and anyone else who enjoys doing Swing work) for since 2008? I have every intention of allowing raw C++ definitions by that mechanism even when EDL is actually a capable language. We'll also want EDL definitions, so  you have a place to put global EDL functions (kind of like a file full of static scripts).

So if the parser takes ten years, fine; do this now, have it just add C++ definitions, and when the parser's done, that's one less fucking thing I have to worry about.

[TheExDeus]

But if I put the code in C++ definitions, then how can I access local or global variables for instances? Like how exactly can I use C++ definitions together with EDL? I guess I can use ds_ functions in the definitions, so I can just make a function that returns a data structure. That is the only thing that comes to mind.

But even then, including vector wasn't an option, because you're still writing code in an event function.

You mean we can't use vectors is because #include would end up in the function? If so, then headers and "using namespace" stuff can be put in the current C++ definitions file. When that is done, then I don't see why the vector wouldn't work inside a script.

[Josh @ Dreamland]

I'm sorry, I couldn't hear you over how great the code in write_object_data.cpp is.

[DarkAceZ]

I'm sorry, I couldn't hear you over how great the code in write_object_data.cpp is.

How can you compete with that, Josh? Why are you still even here? (I bet it's to suggest new shitty site layouts)

[TheExDeus]

Josh wrote at least half of the code. Probably more.

[Goombert]

I'm sorry, I couldn't hear you over how great the code in write_object_data.cpp is.

How can you compete with that, Josh? Why are you still even here? (I bet it's to suggest new shitty site layouts)

In all fairness, mine at least works and does inheritance, Josh's doesn't even work yet.
https://github.com/enigma-dev/enigma-dev/pull/825

[Darkstar2]

Very much agree with you Harri in fact I have a few posts here I made a while ago about some stuff I wanted to do but was told I could not because of the stl containers and certain things not possible in EDL.....  I love your idea to embed native C++ in EDL, bypassing parsing.
There are good uses to this for certain application/games where GM structure is just inefficient.....mind you it's probably efficient for most people using GM lol   but there are certain things I want to do that push the limits, and require optimal speed in file and data handling......and short of writing your own C++ function and integrating it into the ENIGMA engine, this would allow proprietary C++ code outside the engine....

[TheExDeus]

Clearly this feature is way to complicated for the current parser. There is also a possibility this might not work for some reason (something Josh spoke about, but was not willing to elaborate).

[Josh @ Dreamland]

You'd have to access those objects the same way you access them in other C++ functions. Ie, using instance_event_iterator.

The main reason you can't use vector with the current compiler is that it's terrible at coercing types. It was always bad at it, but it's actually gotten worse over time. Originally, you could access vector like an array, but couldn't call functions because the type resolution wasn't good enough to get accurate function information from template types, so I never bothered coding checks for member function calls. I fixed the template types issue in JDI, but in the meantime, someone broke it to the point that it doesn't even realize anything in the code is not a var, anymore. No idea why. Don't feel like investigating.

[TheExDeus]

You'd have to access those objects the same way you access them in other C++ functions. Ie, using instance_event_iterator.

That is why I proposed an inline operator which just ignores parsing. So if I need to access an instance variable, I allow the parser to generate it, like so:

Code: [Select]

ds_grid_set(grid, i, 0, {C++}{myVector.value});So grid will turn into the correct accessor, while myVector.value will not be parsed and be valid C++.

[sorlok_reaves]

I think you'd also need:

Code: [Select]

{C++_instance}{std::vector<int>myVector;}
...where "C++_instance" means "put this in the object declaration for this object type. But what if you declare an integer? Then you'd need:

Code: [Select]

{C++_ctor}{my_var = 10;}
...where "C++_ctor" means "put this in the object's constructor". If you allocate memory, there's a strong argument for "C++ dtor" as well. Oh, and possibly "C++_header", which goes at the top of IDE_EDIT_objectdeclaration.h.

This is one way to do it "without any parsing". We could also parse the code and extract #include<> statements, removing the need for "C++_header"  ...but now we've gotten into parsing C++.

I think this is why everyone is hesitating on the issue ---there's no easy way to hack it in, and (for now) there's no way that's clearly the "right way" to do it.

[TheExDeus]

...where "C++_instance" means "put this in the object declaration for this object type. But what if you declare an integer? Then you'd need:

That is if I want to use those classes or types in several code snippets. I only required the vector<> to be defined inside one function (e.g. a script) and it can be deleted when the function ends. I don't need to declare in create and use in step. So in my case only {C++} would probably work just fine. So the whole idea was that in performance critical script, I can use pure C++, then only copy back the results for local instances. Look at my original code snippet, the myVector is not used outside of that code. It gets deleted when when the script ends. That is why I copy the values back into ds_grid.

If you need something more longer lasting you could use a global variable, but I guess globals are also defined in some declaration and they are not just globally scoped.