Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.

Topics - Goombert

Pages: « 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 »
Third Party / Java 8 Nashorn Script Engine
« on: July 31, 2015, 01:12:40 am »
For a side project I am developing in Java I needed a good JavaScript parser but the publicly documented Nashorn interface is all about compiling when I only needed an intermediate representation. It is currently possible as of JDK8u40 to use the parser to get the AST as a JSON encoded string either from a JS file being executed by the ScriptEngine or from within Java using the non-public Nashorn API.

An abstract syntax tree is produced by a parser after the lexer phase breaks code into a stream of tokens. The below image should convey to you how a simple assignment statement is broken into a stream of tokens then an AST is generated as JSON on the right. This is why symbols like * + - are called binary operators because they take two operands, ! is the logical negation and an unary operator becaues it takes only one operand. The operands can also be expressions because expressions are defined recursively in terms of expressions which can be literals, terms, or other expressions. This is how we end up with tree's and these tree's coupled with semantic information such as keywords and identifiers help us do code generation which can let the whole process take in one language, say GML, and spit out a completely different one like C++ and if you don't already know this is exactly what ENIGMA's compiler does.

Wikipedia has additional information on abstract syntax trees if you would like to know more.
The following StackOverflow post provides clarification between an AST and a parse tree.

My first example here is the standard example on the web of how you can get the JSON tree for any arbitrary JavaScript parsed by a JS script that is being actively executed in Nashorn. You can take this AST and print it or traverse it in JS or pass/return it up to Java through a binding. This example was taken from the following link.
Code: (JavaScript) [Select]
#// Usage: jjs -scripting -fx astviewer.js -- <scriptfile>
 * Copyright (c) 2014, Oracle and/or its affiliates. All rights reserved.
 * Redistribution and use in source and binary forms, with or without
 * modification, are permitted provided that the following conditions
 * are met:
 *   - Redistributions of source code must retain the above copyright
 *     notice, this list of conditions and the following disclaimer.
 *   - Redistributions in binary form must reproduce the above copyright
 *     notice, this list of conditions and the following disclaimer in the
 *     documentation and/or other materials provided with the distribution.
 *   - Neither the name of Oracle nor the names of its
 *     contributors may be used to endorse or promote products derived
 *     from this software without specific prior written permission.

if (!$OPTIONS._fx) {
    print("Usage: jjs -scripting -fx astviewer.js -- <.js file>");

// Using JavaFX from Nashorn. See also:
// This example shows AST of a script file as a JavaFX
// tree view in a window. If no file is specified, AST of
// this script file is shown. This script demonstrates
// 'load' function, JavaFX support by -fx, readFully function
// in scripting mode.

// JavaFX classes used
var StackPane = Java.type("javafx.scene.layout.StackPane");
var Scene     = Java.type("javafx.scene.Scene");
var TreeItem  = Java.type("javafx.scene.control.TreeItem");
var TreeView  = Java.type("javafx.scene.control.TreeView");

// Create a javafx TreeItem to view a AST node
function treeItemForASTNode(ast, name) {
    var item = new TreeItem(name);
    for (var prop in ast) {
       var node = ast[prop];
       if (typeof node == 'object') {
           if (node == null) {
               // skip nulls

           if (Array.isArray(node) && node.length == 0) {
               // skip empty arrays

           var subitem = treeItemForASTNode(node, prop);
       } else {
           var subitem = new TreeItem(prop + ": " + node);
    return item;

// do we have a script file passed? if not, use current script
var sourceName = arguments.length == 0? __FILE__ : arguments[0];

// load parser.js from nashorn resources

// read the full content of the file and parse it
// to get AST of the script specified
var ast = parse(readFully(sourceName));

// JavaFX start method
function start(stage) {
    stage.title = "AST Viewer";
    var rootItem = treeItemForASTNode(ast, sourceName);
    var tree = new TreeView(rootItem);
    var root = new StackPane();
    stage.scene = new Scene(root, 300, 450);;

This example shows you how to get the AST as JSON from Java. This was my own discovery from studying the Nashorn source code.
Code: (Java) [Select]
String code = "function a() { var b = 5; } function c() { }";

Options options = new Options("nashorn");
options.set("anon.functions", true);
options.set("parse.only", true);
options.set("scripting", true);

ErrorManager errors = new ErrorManager();
Context contextm = new Context(options, errors, Thread.currentThread().getContextClassLoader());
String json = ScriptUtils.parse(code, "<unknown>", false);

Both of the above two examples should give the following JSON encoded AST. This JSON encoding provided by Nashorn is compliant with the community standard JavaScript JSON AST model popularized by Mozilla.
Quote from: Java Console

This example code shows you how to get the AST as a Java object representation however the interface is poorly documented and I could not for the life of me figure out how to traverse the children of the function node. This solution is adapted from a StackOverflow post.
Code: (java) [Select]
String code = "function a() { var b = 5; } function c() { }";

// parser options including anonymous functions
final Options options = new Options("nashorn");
options.set("anon.functions", true);
options.set("parse.only", true);
options.set("scripting", true);

ErrorManager errors = new ErrorManager();
Context contextm = new Context(options, errors, Thread.currentThread().getContextClassLoader());
// get a source handle for arbitrary javascript code passed as a string
final Source source = Source.sourceFor("<unknown>", code);

// get the global function node to traverse the parsed AST
FunctionNode node = new Parser(contextm.getEnv(), source, errors).parse();


for (Statement stmt : node.getBody().getStatements()) {

You should get the following output on the Java Console from the above code.
Quote from: Java Console
function {U%}a = [<unknown>] function {U%}a()
function {U%}c = [<unknown>] function {U%}c()

It is important to note however that this interface may change because it's not well documented and is new to the JSE. Additionally the OpenJDK project is developing a public interface for Java 9 that allows AST traversal in a more standard and user friendly way.

Limited documentation for the existing public Nashorn classes in Java 8 can be found below.

The following link provides a list of all of the parser and compiler options that I set above. However it is important to note that the syntax is different when setting the options inside Java where - is replaced with a period.

The Nashorn source code can be found on GitHub and also on BitBucket. I prefer the BitBucket version as the GitHub version seems to be missing some classes.

Developing ENIGMA / MinGW 64
« on: January 17, 2015, 12:41:35 pm »
Ok so I decided to see what it would take to finally support 64bit in ENIGMA.

The first thing was getting JDI and the Compiler to build with MinGW64

Edit: Josh approves as of this commit.

Edit: Josh advised me to use the fixed size types for the overload, so now the compiler builds with C++11 support too for both 32bit and 64bit GCC

I should not have wrote those straight to master but they may need reverted, anyway...

64bit Java/JNA can only load 64bit dll's and 32bit Java/JNA can only load 32bit dll's. This means that the users Java installation with ENIGMA thanks to the plugin and using ENIGMA the way we do will have to match the architecture of that the user not only has but also intends to target.

So if you have 64bit Java, with any ENIGMA release you will only be able to make 64bit games. And if you have 32bit Java you can only build and use the 32bit version of ENIGMA. But you could install both 32bit and 64bit ENIGMA in parallel and Java in parallel to be able to build for both architectures if you have a 64bit OS.

One possible alternative is to remove the requirement of JNA and compileEGMf all together replacing it entirely with the CLI allowing you to build both 32bit and 64bit applications with either a 32bit or 64bit Java installation. We could then use a MinGW that supports dual target though the exception support would be pretty grotesque, just because that's the state of current dual target MinGW compilers. Another way of accomplishing the same thing is to build compileEGMf for the supported Java architectures which would still require a dual target MinGW.

1) Do we want both a 32 bit and 64 bit ENIGMA portable, or just a very large single ENIGMA portable? If the former you would have to download both portables in order to build for both architectures.
2) Do we want to be able to build 32 bit games when we have a 64 bit Java installed to run LGM?
3) This is tied to the first question, but do you want proper gcc exception support or not? Because if we go with dual target either 64bit exception support is bad or we maintain two separate releases. If not we could include both mingw32 and mingw64 instead of a dual target mingw64, and that may or may not mean we have two separate portables.

Addressing this problem would fix several issues.
1) 64bit Java would be supported to run ENIGMA
2) 64bit compilation support would be added.

For the record Qt Framework also makes you install separately, as does Java and .NET and a lot of other programs.

Note: The good news is also that JNA supports both 32bit and 64bit so there's no need to distribute two versions of it.

Proposals / Disabling Automatic Semicolons
« on: December 30, 2014, 12:55:38 am »
After I fixed the primitive arrays with Josh's help I wanted to fix initializer lists, they are only broken because of the automatic semicolons, when this is disabled they work. Harri's matrix array initialization is primarily what inspired me.

This was my attempt here:

But it ultimately fails because we simply don't have sufficient information to do this yet and a number of games are failing to parse after the changes. A better work around for the time being would be to add an option to disable automatic semicolons all together, this means you would have to always put your ';' terminating semicolons where they belong. But it would stop some things like initializer lists from breaking.

This would make the following possible in ENIGMA if you choose to disable it.
Code: (EDL) [Select]
int arr[2][3][4] = { { {1, 2, 3, 4}, {1, 2, 3, 4}, {1, 2, 3, 4} },
                     { {1, 2, 3, 4}, {1, 2, 3, 4}, {1, 2, 3, 4} } };

for (int r = 0; r < 2; r++) {
    for (int c = 0; c < 3; c++) {
        for (int n = 0; n < 4; n++) {

I would appreciate feedback on this and whether you guys think a setting for this would be nice. It should be considered an advanced setting as advanced users are more likely to properly terminate statements.

Third Party / Porting GMOgre
« on: December 29, 2014, 06:14:05 am »
Well I've got some good news, I decided to try porting GMOgre again and was met with relative success. You can download the original examples from the GMC for now, I have not made a special ENIGMA release yet.

To start off I had to do similar fixes that were required for Ultimate3D, the window flags fix from a while back that has been added in the latest Portable ZIP is also needed.

Here are the problems:
1) Global variable exists function used to check if OGRE is initialized this, I just replaced this with global.ogre_initialized == true
2) Local variable exists functions used to initialize z values for objects, replaced this with a local flag and enabled "Treat unitialized variables as 0" in Game Settings->Errors
3) ENIGMA has not implemented temp_directory for the settings.ini to be stored, so I replaced it with working_directory, which is where the dll also must be kept
4) The GMOgre project files are totally corrupt, I had to import into Studio and then use LGM to convert the GMX back into GMK. The problem is the resources have their names but the part of the GMK storing the tree has them with the wrong names, just garbage. The GMK's also have problems in GM8.1
5) There's a conflict with the script named CreateFont, so I had to rename it to CreateFontA
6) There's a bug with default script arguments in obj_skybox create event. It calls EnableSkybox passing only argument1 skipping argument0 and 2-3. This is in fact a mistake in GMOgre, not an issue with ENIGMA, I tested GM8.1 and it does not allow skipping arguments, and neither does ISO C.
7) The OGRE log reports several shaders not being compiled, if we add the following code:
Code: (GML) [Select]
To the create event of obj_engine then different errors occur. I created these logs after changing the renderer to GL from DX9
8) We have no way of supporting GMAPI, this is why the RenderFrame script crashes, it is trying to call GML functions using GMAPI which Studio no longer supports either.

After fixing those problems I managed to get the GMOgre FPS example built but it crashes right after starting and throws debug messages about undefined vars.

This is the backtrace from GDB, if anybody has any ideas let me know.
Code: [Select]
(gdb) bt
#0  0x100d9f76 in ?? ()
#1  0x008c04c3 in ffi_call (cif=<optimized out>, fn=<optimized out>,
    rvalue=<optimized out>, avalue=<optimized out>)
    at /root/enigger_libs/mingw-w64-libffi/src/libffi-3.0.13/src/x86/ffi.c:405
#2  0x007b1a08 in enigma_user::external_call (id=497, a1=..., a2=..., a3=...,
    a4=..., a5=..., a6=..., a7=..., a8=..., a9=..., a10=..., a11=...,
    a12=..., a13=..., a14=..., a15=..., a16=...)
    at Platforms/Win32/WINDOWSexternals.cpp:176
#3  0x005f0a9e in _SCR_RenderFrame (argument0=..., argument1=...,
    argument2=..., argument3=..., argument4=..., argument5=...,
    argument6=..., argument7=..., argument8=..., argument9=...,
    argument10=..., argument11=..., argument12=..., argument13=...,
    argument14=..., argument15=...)
    at C:/ProgramData/ENIGMA/Preprocessor_Environment_Editable/IDE_EDIT_objectfu
#4  0x0077c724 in enigma::OBJ_obj_engine::myevent_endstep (this=0x3c26140)
    at C:/ProgramData/ENIGMA/Preprocessor_Environment_Editable/IDE_EDIT_objectfu
#5  0x00424ae6 in enigma::ENIGMA_events ()
    at C:/ProgramData/ENIGMA/Preprocessor_Environment_Editable/IDE_EDIT_events.h
#6  0x007a89a1 in WinMain@16 (hInstance=0x400000, hPrevInstance=0x0,
    lpCmdLine=0x238479f "", iCmdShow=10)
    at Platforms/Win32/WINDOWSmain.cpp:356
#7  0x00b2871d in main ()

Developing ENIGMA / New Portable
« on: December 28, 2014, 12:10:58 am »
We've had some really awesome compiler fixes lately and I wanted to get these fixes out to everyone for testing. I have not updated LateralGM or the plugin since the last Portable ZIP, this was just a quick releases for these compiler fixes.

You can update by downloading the new ZIP.

You can also get these changes by entering the following in a terminal or using git-bash.exe
Code: [Select]
cd enigma-dev
git fetch
git pull

1) Nested macros have been fixed, this is basically nested string() calls, they will now work.
2) Fixed primitive arrays adding full multi-dimensional primitive array functionality.
3) Finished implementing the modulus assignment (%=) operator and overloaded it for var
4) Fixed alarm inheritance, though alarms may still fire out of order, they are supposed to be fired 0,1,2,3,etc. but currently ENIGMA fires the parents then the childs, this will be fixed in the future, but it is unlikely to cause a bug and it still works better than before. I can't really think of an example where someone would rely on alarm 1 to fire after alarm 0.
5) Instance activation/deactivation when used with inheritance fixed by sorlok
6) Enabled Unicode RC files, so you can enter the copyright symbol into LGM's platform settings now and it will properly encode it in your game executable
7) Fixes syntax checking output, fixes the line numbers and also actually formats the output for script, instance and room creation scope where it did not before.

General ENIGMA / Extension Depends on Extension?
« on: December 13, 2014, 02:01:31 am »
Not sure about this one but I'd like to know why the following isn't working for me.

Code: (yaml) [Select]
  Extensions: DataStructures

My asynchronous extension relies on the Data Structure extension and needs it to be compiled first otherwise a segfault ensues. Since I can't find another extension that relies on another extension as an example I have absolutely nothing to go on and the Wiki page doesn't distinguish.

Do we have this system in place yet or not? Otherwise I will have to hard code the ds extension into the async extension and you'll have to build without the DS extension enabled.

Tips, Tutorials, Examples / Advanced Platform Example
« on: December 10, 2014, 12:44:11 am »
This is just a heads up I ran across the tutorial for GM6 over at the GMC and it works perfectly in ENIGMA with no changes.

You can follow the tutorial from start to finish with ENIGMA.

General ENIGMA / New Portable
« on: December 10, 2014, 12:04:40 am »

I decided to build a new Portable, it's been quite a while and the old one has gotten stale, plus most of the changes have been pretty stable and well received.

You can get it from the Windows install page.

This release includes several of the latest LGM and plugin changes such as all the awesome searching in resources features and what not, you can check the other topic for that info.

Let me point out some specific engine/compiler changes brought by this ZIP release:
1) Window flags were fixed to make extensions like Ultimate3D work, they are exactly the same flags used by GM8's window.
2) Object writing was refactored to use an instance tree by Josh, this makes several other future compiler fixes much easier.
3) With(self) was fixed by sorlok which makes Polygonz Sonic engine work again
4) instance_change fix provided by sorlok
5) Surface fixes and other GL changes by Harri
6) instance_deactivate_object fix by sorlok
7) Timelines infinite loop fixed by sorlok
8) Persistence memory leak fixed by Josh
9) Array length functionality and var overloading implemented by sorlok

There's been a ton of other changes but I don't see them as that important to list here, if you think one should be then tell me.

General ENIGMA / Code Action Comments
« on: December 07, 2014, 06:41:41 pm »
This has been in Studio for a while, you can use /// on the first line of code action to change its descriptive label. I just want to know what everyone thinks of this feature, maybe we could add it to LGM or not, would just like to know what people think.

From the GMC:

Off-Topic / Contributor Status
« on: November 04, 2014, 09:03:55 pm »
I would like take to a moment and welcome sorlok and egofree both to official contributor status on the forums because of their awesome contributions to the project. You have both been immensely helpful to me, the project, and other users and forum members here. I like having you both as contributors of LGM and ENIGMA, you are both extremely personable and reasonable people and it has been a pleasure working with you both.

Congratulations, you've earned the title!  :D

Note: New assignments to git privileges are not being administered at this time.

Off-Topic / Windows 10 Package Manager
« on: October 29, 2014, 03:20:42 pm »
Well well well. Windows 10 will have a package manager that will make Windows development just amazing, we may finally be able to create a proper Portable ZIP and allow MinGW installations. This is honestly the greatest news ever!

Even those of you who want to go ahead and start using PowerShell can, I was only vaguely aware of its existence, it has rectangle select as well. The only downside is that it is built on the .NET framework, it opens at about 45 MB ram usage for me where as regular cmd opens at 0.3 MB.

Tips, Tutorials, Examples / GameMaker 8.1 Icons
« on: October 29, 2014, 01:12:53 pm »
I used the following tool to extract the icons from GameMaker 8.1, and you will need to download it in order to do this yourself.

Follow these steps to create the icon pack.
1) Open LateralGM and set the icon pack to "Custom"
2) Set the folder path for the custom icon pack if you wish, by default I will just leave it to the icons folder next to lateralgm.jar
3) Close LateralGM
4) Create the folder called "icons" from step (2) next to lateralgm.jar, usually in the enigma-dev folder
5) Copy your icons to this folder in the correct structure, e.g, using the same filenames as LGM does. Look at the Calico built-in icon pack as an example, you will need to organize the icons into the folders "actions", "events", and "restree" inside your icons folder.
5) Start LateralGM and the icon pack should have loaded

Off-Topic / Windows 8 Virtual Desktops
« on: October 20, 2014, 02:59:38 am »
No joke, it's been built into Windows since XP, and Microsoft has an installer for you to access the hidden feature, so it's not a rough hack.

General ENIGMA / Who fixed arrays?
« on: October 10, 2014, 02:42:33 am »
I was going to do a little something, and realized that arrays were fixed. I can't quite recall who or what fixed them or when they did it, but they do seem to work now.

The following builds fine for me on the latest master.
Code: (EDL) [Select]
var ass;
ass[0] = 69;


Whoever it was, thank you!

Additionally that thing I was trying to do was provide an array length function, but sadly JDI fails to parse the templates, it keeps saying the function is undefined unless I change the parameter
Code: (cpp) [Select]
  template <unsigned array_size>
  unsigned array_length_1d(variant (&v)[array_size]);

Off-Topic / BlitzBasic Gone Free and Open Source!
« on: October 10, 2014, 02:12:13 am »
I've always been a big fan of the BlitzBasic engine and products, though never really having used them that much, I liked the environment much better over GM. It made it very easy to manage objects and everything from code and for novices to learn without having to create excessive GUI infrastructures. Well anyway since Mark Sibly is focused on Monkey X now, they've put Blitz Plus and Blitz 3D up for free. Blitz Max and the programming manual are still being commercially sold.

You can read the official announcements on the home page and the forum threads.

Anybody not already aware, Monkey X is an open source cross-platform game engine with the BASIC programming language as well, sort of based on BlitzBasic. For $100 it can export to numerous modules including Android, Playstation, Xbox, and other platforms. The IDE is built, quite evidently, with the Qt Framework, and it's a very nice IDE.

Pages: « 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 »