How can I use large arrays more than 30 MB?

[Darkstar2]

Code: (gml) [Select]

var a;
int i;
size = 40000000;
for (i = 0; i < size; i++)
{
a[i] = 65;
}

This would compile without error but crash. Stack overflow I guess.

However if I were to use 30000000 it would work fine.

So 30MB seems to be the limit.

How can I modify the code above to make it work.
In C++ for these things one would use the heap
and malloc, but I can't do that from within ENIGMA.

Any way to solve this?

This would mean I cannot load a resource more than 30MB in the IDE ?? that is crap.! 

[Josh @ Dreamland]

Interesting. My guess is that you're actually exhausting the 2GB of RAM Windows allows a 32-bit program to use. You aren't allocating 30MB, you are allocating 30 million entries. Even if each entry was only one double, that would actually be 240MB of usage. Issue is, you're not just allocating one double for each entry. You're allocating a double and a string, which is a pointer to a reference count, a length, and a few bytes of string. So now we're at 8+4+4+4+1 bytes, being extremely conservative. This puts you at 840MB. Now, here's the kicker: since your array is dense, and the size is not given up front, I have to resize it every time you overflow the space I allocate you. So by the time you hit the 32 millionth entry or so, I need to keep your array around, and then allocate a new array big enough to hold that array and then arbitrarily more information. There's just no way for this allocation to work inside of a 2GB addressing space.

Try putting this in the create event:

Code: (edl) [Select]

local size_t size = 40000000;
local int *array;
array = new int[size];

for (int i = 0; i < size; ++i) {
  array[i] = 65;
}

Also, in the destroy event, put [snip=edl]delete[] array;[/snip].

[Goombert]

Allocated on the heap, automatic storage duration does not apply to some pointers but does to most of STL and generic collections, just ignore me I'm doing this for my own good so that I don't forget.

[TheExDeus]

Josh, is that valid EDL now? Previously most of that stuff didn't work. I'll check in master.

Darkstar2, as Josh mentioned the problem is you probably end up using more than 2GB of ram. Look at task manager. Normally you would be able to do it like this:

Code: (edl) [Select]

size = 40000000;
local variant a[40000000];
for (int i = 0; i < size; ++i)
{
        a[i] = 65;
}But sadly the parser doesn't support this yet.

edit: Jup, Josh it doesn't work for the same reason my posted code doesn't work - "a[ i ] = 65" parses as "a(i) = 65", and so arrays like these don't work. That is why we are waiting for your parser sooooo patiently.

[Josh @ Dreamland]

Finding the motivation to finish this has proven problematic. Why don't you appoint someone to wrestling their way through my task list?

I'll polish up the output of whatever poor soul attempts it.

Just so everyone knows: This is a joke. I don't really trust any of you for the task. Maybe today I'll muddle through this lexer recode.

[Darkstar2]

Thanks

I have an idea, if this doesn't parse through EDL,
how about I make those in C++ directly, wrap it inside
a function inside namespace enigma and call it from EDL, would that work?