Java 8 Nashorn Script Engine

[Goombert]

For a side project I am developing in Java I needed a good JavaScript parser but the publicly documented Nashorn interface is all about compiling when I only needed an intermediate representation. It is currently possible as of JDK8u40 to use the parser to get the AST as a JSON encoded string either from a JS file being executed by the ScriptEngine or from within Java using the non-public Nashorn API.

An abstract syntax tree is produced by a parser after the lexer phase breaks code into a stream of tokens. The below image should convey to you how a simple assignment statement is broken into a stream of tokens then an AST is generated as JSON on the right. This is why symbols like * + - are called binary operators because they take two operands, ! is the logical negation and an unary operator becaues it takes only one operand. The operands can also be expressions because expressions are defined recursively in terms of expressions which can be literals, terms, or other expressions. This is how we end up with tree's and these tree's coupled with semantic information such as keywords and identifiers help us do code generation which can let the whole process take in one language, say GML, and spit out a completely different one like C++ and if you don't already know this is exactly what ENIGMA's compiler does.



Wikipedia has additional information on abstract syntax trees if you would like to know more.
https://en.wikipedia.org/wiki/Abstract_syntax_tree
The following StackOverflow post provides clarification between an AST and a parse tree.
http://stackoverflow.com/questions/5026517/whats-the-difference-between-parse-tree-and-ast

My first example here is the standard example on the web of how you can get the JSON tree for any arbitrary JavaScript parsed by a JS script that is being actively executed in Nashorn. You can take this AST and print it or traverse it in JS or pass/return it up to Java through a binding. This example was taken from the following link.
http://hg.openjdk.java.net/jdk8u/jdk8u-dev/nashorn/file/bfea11f8c8f2/samples/astviewer.js

Code: (JavaScript) [Select]

#// Usage: jjs -scripting -fx astviewer.js -- <scriptfile>
/*
 * Copyright (c) 2014, Oracle and/or its affiliates. All rights reserved.
 *
 * Redistribution and use in source and binary forms, with or without
 * modification, are permitted provided that the following conditions
 * are met:
 *
 *   - Redistributions of source code must retain the above copyright
 *     notice, this list of conditions and the following disclaimer.
 *
 *   - Redistributions in binary form must reproduce the above copyright
 *     notice, this list of conditions and the following disclaimer in the
 *     documentation and/or other materials provided with the distribution.
 *
 *   - Neither the name of Oracle nor the names of its
 *     contributors may be used to endorse or promote products derived
 *     from this software without specific prior written permission.
 *
 * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
 * IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO
 * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
 * PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL THE COPYRIGHT OWNER OR
 * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
 * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
 * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
 * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
 * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
 * NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
 * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 */

if (!$OPTIONS._fx) {
    print("Usage: jjs -scripting -fx astviewer.js -- <.js file>");
    exit(1);
}

// Using JavaFX from Nashorn. See also:
// http://docs.oracle.com/javase/8/docs/technotes/guides/scripting/nashorn/javafx.html
// This example shows AST of a script file as a JavaFX
// tree view in a window. If no file is specified, AST of
// this script file is shown. This script demonstrates
// 'load' function, JavaFX support by -fx, readFully function
// in scripting mode.

// JavaFX classes used
var StackPane = Java.type("javafx.scene.layout.StackPane");
var Scene     = Java.type("javafx.scene.Scene");
var TreeItem  = Java.type("javafx.scene.control.TreeItem");
var TreeView  = Java.type("javafx.scene.control.TreeView");

// Create a javafx TreeItem to view a AST node
function treeItemForASTNode(ast, name) {
    var item = new TreeItem(name);
    for (var prop in ast) {
       var node = ast[prop];
       if (typeof node == 'object') {
           if (node == null) {
               // skip nulls
               continue;
           }

           if (Array.isArray(node) && node.length == 0) {
               // skip empty arrays
               continue;
           }

           var subitem = treeItemForASTNode(node, prop);
       } else {
           var subitem = new TreeItem(prop + ": " + node);
       }
       item.children.add(subitem);
    }
    return item;
}


// do we have a script file passed? if not, use current script
var sourceName = arguments.length == 0? __FILE__ : arguments[0];


// load parser.js from nashorn resources
load("nashorn:parser.js");


// read the full content of the file and parse it
// to get AST of the script specified
var ast = parse(readFully(sourceName));


// JavaFX start method
function start(stage) {
    stage.title = "AST Viewer";
    var rootItem = treeItemForASTNode(ast, sourceName);
    var tree = new TreeView(rootItem);
    var root = new StackPane();
    root.children.add(tree);
    stage.scene = new Scene(root, 300, 450);
    stage.show();
}

This example shows you how to get the AST as JSON from Java. This was my own discovery from studying the Nashorn source code.

Code: (Java) [Select]

String code = "function a() { var b = 5; } function c() { }";

Options options = new Options("nashorn");
options.set("anon.functions", true);
options.set("parse.only", true);
options.set("scripting", true);

ErrorManager errors = new ErrorManager();
Context contextm = new Context(options, errors, Thread.currentThread().getContextClassLoader());
Context.setGlobal(contextm.createGlobal());
String json = ScriptUtils.parse(code, "<unknown>", false);
System.out.println(json);

Both of the above two examples should give the following JSON encoded AST. This JSON encoding provided by Nashorn is compliant with the community standard JavaScript JSON AST model popularized by Mozilla.
https://developer.mozilla.org/en-US/docs/Mozilla/Projects/SpiderMonkey/Parser_API

{"type":"Program","body":[{"type":"FunctionDeclaration","id":{"type":"Identifier","name":"a"},"params":[],"defaults":[],"rest":null,"body":{"type":"BlockStatement","body":[{"type":"VariableDeclaration","declarations":[{"type":"VariableDeclarator","id":{"type":"Identifier","name":"b"},"init":{"type":"Literal","value":5}}]}]},"generator":false,"expression":false},{"type":"FunctionDeclaration","id":{"type":"Identifier","name":"c"},"params":[],"defaults":[],"rest":null,"body":{"type":"BlockStatement","body":[]},"generator":false,"expression":false}]}

This example code shows you how to get the AST as a Java object representation however the interface is poorly documented and I could not for the life of me figure out how to traverse the children of the function node. This solution is adapted from a StackOverflow post.
http://stackoverflow.com/questions/6511556/javascript-parser-for-java

Code: (java) [Select]

String code = "function a() { var b = 5; } function c() { }";

// parser options including anonymous functions
final Options options = new Options("nashorn");
options.set("anon.functions", true);
options.set("parse.only", true);
options.set("scripting", true);

ErrorManager errors = new ErrorManager();
Context contextm = new Context(options, errors, Thread.currentThread().getContextClassLoader());
// get a source handle for arbitrary javascript code passed as a string
final Source source = Source.sourceFor("<unknown>", code);

// get the global function node to traverse the parsed AST
FunctionNode node = new Parser(contextm.getEnv(), source, errors).parse();

System.out.println(node.getKind().name());

for (Statement stmt : node.getBody().getStatements()) {
System.out.println(stmt.toString());
}

You should get the following output on the Java Console from the above code.

SCRIPT
function {U%}a = [<unknown>] function {U%}a()
function {U%}c = [<unknown>] function {U%}c()

It is important to note however that this interface may change because it's not well documented and is new to the JSE. Additionally the OpenJDK project is developing a public interface for Java 9 that allows AST traversal in a more standard and user friendly way.
http://openjdk.java.net/jeps/236

Limited documentation for the existing public Nashorn classes in Java 8 can be found below.
https://docs.oracle.com/javase/8/docs/jdk/api/nashorn/allclasses-noframe.html

The following link provides a list of all of the parser and compiler options that I set above. However it is important to note that the syntax is different when setting the options inside Java where - is replaced with a period.
http://hg.openjdk.java.net/jdk8u/jdk8u-dev/nashorn/file/tip/docs/DEVELOPER_README

The Nashorn source code can be found on GitHub and also on BitBucket. I prefer the BitBucket version as the GitHub version seems to be missing some classes.
https://github.com/uditrugman/openjdk8-nashorn
https://bitbucket.org/adoptopenjdk/jdk8-nashorn/src/096dc407d310?at=default