让ANTLR生成脚本解释器?

Say I have the following Java API that all packages up as blocks.jar:

假设我有以下的Java API，所有的包都是块。

public class Block {
    private Sting name;
    private int xCoord;
    private int yCoord;

    // Getters, setters, ctors, etc.

    public void setCoords(int x, int y) {
        setXCoord(x);
        setYCoord(y);
    }
}

public BlockController {
    public static moveBlock(Block block, int newXCoord, int newYCoord) {
        block.setCooords(newXCoord, newYCoord);
    }

    public static stackBlocks(Block under, Block onTop) {
        // Stack "onTop" on top of "under".
        // Don't worry about the math here, this is just for an example.
        onTop.setCoords(under.getXCoord() + onTop.getXCoord(), under.getYCoord());
    }
}

Again, don't worry about the math and the fact that (x,y) coordinates don't accurately represent blocks in 3D space. The point is that we have Java code, compiled as a JAR, that performs operations on blocks. I now want to build a lightweight scripting language that allows a non-programmer to invoke the various block API methods and manipulate blocks, and I want to implement its interpreter with ANTLR (latest version is 4.3).

同样，不要担心数学和(x,y)坐标不能准确地表示三维空间中的块。关键是我们有Java代码，作为JAR编译，在块上执行操作。我现在想构建一种轻量级的脚本语言，允许非程序员调用各种块API方法并操作块，我还想使用ANTLR实现它的解释器(最新版本是4.3)。

The scripting language, we'll call it BlockSpeak, might look like this:

脚本语言，我们称之为BlockSpeak，可能是这样的:

block A at (0, 10)   # Create block "A" at coordinates (0, 10)
block B at (0, 20)   # Create block "B" at coordinates (0, 20)
stack A on B         # Stack block A on top of block B

This might be equivalent to the following Java code:

这可能等价于以下Java代码:

Block A, B;
A = new Block(0, 10);
B = new Block(0, 20);
BlockController.stackBlocks(B, A);

So the idea is that the ANTLR-generated interpreter would take a *.blockspeak script as input, and use the commands in this script to invoke blocks.jar API operations. I read the excellent Simple Example which creates a simple calculator using ANTLR. However in that link, there is an ExpParser class with an eval() method:

这个想法是由antlr生成的解释器取一个*。blockspeak脚本作为输入，并使用该脚本中的命令来调用块。jar API操作。我阅读了一个优秀的简单示例，它使用ANTLR创建了一个简单的计算器。但是在该链接中，有一个带有eval()方法的ExpParser类:

ExpParser parser = new ExpParser(tokens);
parser.eval();

The problem here is that, in the case of the calculator, the tokens represent a mathematical expression to evaluate, and eval() returns the evaluation of the expression. In the case of an interpreter, the tokens would represent my BlockSpeak script, but calling eval() shouldn't evaluate anything, it should know how to map the various BlockSpeak commands to Java code:

这里的问题是，在计算器的例子中，标记表示要求值的数学表达式，eval()返回表达式的求值。在解释器的情况下，令牌将代表我的BlockSpeak脚本，但是调用eval()不应该评估任何东西，它应该知道如何将各种BlockSpeak命令映射到Java代码:

BlockSpeak Command:             Java code:
==========================================
block A at (0, 10)      ==>     Block A = new Block(0, 10);
block B at (0, 20)      ==>     Block B = new Block(0, 20);
stack A on B            ==>     BlockController.stackBlocks(B, A);

So my question is, where do I perform this "mapping"? In other words, how do I instruct ANTLR to call various pieces of code (packaged inside blocks.jar) when it encounters particular grammars in the BlockSpeak script? More importantly, can someone give me a pseudo-code example?

我的问题是，我在哪里执行这个映射?换句话说，当ANTLR在BlockSpeak脚本中遇到特定的语法时，如何指示它调用不同的代码片段(封装在block .jar中)?更重要的是，有人能给我一个伪代码示例吗?

3 个解决方案

#1

I would simply evaluate the script on the fly, not generate Java source files which need to be compiled themselves again.

我只是动态地评估脚本，而不是生成需要重新编译的Java源文件。

With ANTLR 4 it is highly recommended to keep the grammar and target specific code separate from each other and put any target specific code inside a tree-listener or -visitor.

对于ANTLR 4，强烈建议将语法和目标特定的代码分开，并将任何目标特定的代码放在树监听器或访问者中。

I will give a quick demo how to use a listener.

我将快速演示如何使用侦听器。

A grammar for your example input could look like this:

示例输入的语法可以如下所示:

File: blockspeak/BlockSpeak.g4

grammar BlockSpeak;

parse
 : instruction* EOF
 ;

instruction
 : create_block
 | stack_block
 ;

create_block
 : 'block' NAME 'at' position
 ;

stack_block
 : 'stack' top=NAME 'on' bottom=NAME
 ;

position
 : '(' x=INT ',' y=INT ')'
 ;

COMMENT
 : '#' ~[\r\n]* -> skip
 ;

INT
 : [0-9]+
 ;

NAME
 : [a-zA-Z]+
 ;

SPACES
 : [ \t\r\n] -> skip
 ;

Some supporting Java classes:

一些支持Java类:

File: blockspeak/Main.java

package blockspeak;

import org.antlr.v4.runtime.ANTLRInputStream;
import org.antlr.v4.runtime.CommonTokenStream;
import org.antlr.v4.runtime.tree.ParseTreeWalker;

import java.util.Scanner;

public class Main {

    public static void main(String[] args) throws Exception {

        Scanner keyboard = new Scanner(System.in);

        // Some initial input to let the parser have a go at.
        String input = "block A at (0, 10)   # Create block \"A\" at coordinates (0, 10)\n" +
                "block B at (0, 20)   # Create block \"B\" at coordinates (0, 20)\n" +
                "stack A on B         # Stack block A on top of block B";

        EvalBlockSpeakListener listener = new EvalBlockSpeakListener();

        // Keep asking for input until the user presses 'q'.
        while(!input.equals("q")) {

            // Create a lexer and parser for `input`.
            BlockSpeakLexer lexer = new BlockSpeakLexer(new ANTLRInputStream(input));
            BlockSpeakParser parser = new BlockSpeakParser(new CommonTokenStream(lexer));

            // Now parse the `input` and attach our listener to it. We want to reuse 
            // the same listener because it will hold out Blocks-map.
            ParseTreeWalker.DEFAULT.walk(listener, parser.parse());

            // Let's see if the user wants to continue.
            System.out.print("Type a command and press return (q to quit) $ ");
            input = keyboard.nextLine();
        }

        System.out.println("Bye!");
    }
}

// You can place this Block class inside Main.java as well.
class Block {

    final String name;
    int x;
    int y;

    Block(String name, int x, int y) {
        this.name = name;
        this.x = x;
        this.y = y;
    }

    void onTopOf(Block that) {
        // TODO
    }
}

This main class is pretty self explanatory with the inline comments. The tricky part is what the listener is supposed to look like. Well, here it is:

使用内联注释，这个主类是很容易理解的。棘手的部分是听众应该是什么样子。嗯,这里是:

File: blockspeak/EvalBlockSpeakListener.java

package blockspeak;

import org.antlr.v4.runtime.misc.NotNull;

import java.util.HashMap;
import java.util.Map;

/**
 * A class extending the `BlockSpeakBaseListener` (which will be generated
 * by ANTLR) in which we override the methods in which to create blocks, and
 * in which to stack blocks.
 */
public class EvalBlockSpeakListener extends BlockSpeakBaseListener {

    // A map that keeps track of our Blocks.
    private final Map<String, Block> blocks = new HashMap<String, Block>();

    @Override
    public void enterCreate_block(@NotNull BlockSpeakParser.Create_blockContext ctx) {

        String name = ctx.NAME().getText();
        Integer x = Integer.valueOf(ctx.position().x.getText());
        Integer y = Integer.valueOf(ctx.position().y.getText());

        Block block = new Block(name, x, y);

        System.out.printf("creating block: %s\n", name);

        blocks.put(block.name, block);
    }

    @Override
    public void enterStack_block(@NotNull BlockSpeakParser.Stack_blockContext ctx) {

        Block bottom = this.blocks.get(ctx.bottom.getText());
        Block top = this.blocks.get(ctx.top.getText());

        if (bottom == null) {
            System.out.printf("no such block: %s\n", ctx.bottom.getText());
        }
        else if (top == null) {
            System.out.printf("no such block: %s\n", ctx.top.getText());
        }
        else {
            System.out.printf("putting %s on top of %s\n", top.name, bottom.name);
            top.onTopOf(bottom);
        }
    }
}

The listener above has 2 methods defined that map to the following parser rules:

上面的侦听器有两个方法，它们映射到以下解析器规则:

create_block
 : 'block' NAME 'at' position
 ;

stack_block
 : 'stack' top=NAME 'on' bottom=NAME
 ;

Whenever the parser "enters" such a parser rule, the corresponding method inside the listener will be called. So, whenever enterCreate_block (the parser enters the create_block rule) is called, we create (and save) a block, and when enterStack_block is called, we retrieve the 2 block involved in the operation, and stack one on top of the other.

每当解析器“输入”这样的解析器规则时，将调用侦听器内的相应方法。因此，每当调用enterCreate_block(解析器进入create_block规则)时，我们创建(并保存)一个块，当调用enterStack_block时，我们检索操作中涉及的两个块，并将一个块叠加在另一个块上。

To see the 3 classes above in action, download ANTLR 4.4 inside the directory that holds the blockspeak/ directory with the .g4 and .java files.

要查看上面的3个类的实际操作，请在包含.g4和.java文件的blockspeak/目录的目录中下载ANTLR 4.4。

Open a console and perform the following 3 steps:

打开控制台，执行以下三个步骤:

1. generate the ANTLR files:

java -cp antlr-4.4-complete.jar org.antlr.v4.Tool blockspeak/BlockSpeak.g4 -package blockspeak

2. compile all Java sources files:

javac -cp ./antlr-4.4-complete.jar blockspeak/*.java

3. Run the main class:

3.1. Linux/Mac

java -cp .:antlr-4.4-complete.jar blockspeak.Main

3.2. Windows

java -cp .;antlr-4.4-complete.jar blockspeak.Main

Here is an example session of running the Main class:

下面是一个运行主类的示例会话:

bart@hades:~/Temp/demo$ java -cp .:antlr-4.4-complete.jar blockspeak.Main
creating block: A
creating block: B
putting A on top of B
Type a command and press return (q to quit) $ block X at (0,0)
creating block: X
Type a command and press return (q to quit) $ stack Y on X
no such block: Y
Type a command and press return (q to quit) $ stack A on X 
putting A on top of X
Type a command and press return (q to quit) $ q
Bye!
bart@hades:~/Temp/demo$ 

More info on tree listeners: https://theantlrguy.atlassian.net/wiki/display/ANTLR4/Parse+Tree+Listeners

关于树监听器的更多信息:https://theantlrguy.atlassian.net/wiki/display/antl4/parse + tree +监听器

#2

I would personally write a grammar to generate a Java program for each script that you could then compile (along with your jar) and run independently... i.e., a 2-step process.

我个人会编写一个语法来为每个脚本生成一个Java程序，然后您可以编译(以及您的jar)并独立运行……即。,一个两步的过程。

For example, with something like the following simple grammar (which I haven't tested and I am sure you would need to extend and adapt), you could replace the parser.eval() statement in that example with parser.program(); (also substituting "BlockSpeak" for "Exp" throughout) and it should spit out Java code that matches the script to stdout, which you could redirect into a .java file, compile (together with the jar) and run.

例如，使用如下这样的简单语法(我还没有测试过，我确信您需要扩展和调整)，您可以将示例中的parser.eval()语句替换为parser.program();(还将“BlockSpeak”替换为“Exp”)并且它应该输出与脚本匹配的Java代码到stdout，您可以将这些代码重定向到. Java文件中，编译(与jar一起)并运行。

BlockSpeak.g:

BlockSpeak.g:

grammar BlockSpeak;

program 
    @init { System.out.println("//import com.whatever.stuff;\n\npublic class BlockProgram {\n    public static void main(String[] args) {\n\n"); }
    @after { System.out.println("\n    } // main()\n} // class BlockProgram\n\n"); }
    : inss=instructions                         { if (null != $inss.insList) for (String ins : $inss.insList) { System.out.println(ins); } }
    ;

instructions returns [ArrayList<String> insList]
    @init { $insList = new ArrayList<String>(); }
    : (instruction { $insList.add($instruction.ins); })* 
    ;

instruction returns [String ins]
    :  ( create { $ins = $create.ins; } | move  { $ins = $move.ins; } | stack { $ins = $stack.ins; } ) ';' 
    ;

create returns [String ins]
    :  'block' id=BlockId 'at' c=coordinates    { $ins = "        Block " + $id.text + " = new Block(" + $c.coords + ");\n"; }
    ;

move returns [String ins]
    :  'move' id=BlockId 'to' c=coordinates     { $ins = "        BlockController.moveBlock(" + $id.text + ", " + $c.coords + ");\n"; }
    ;

stack returns [String ins]
    :  'stack' id1=BlockId 'on' id2=BlockId     { $ins = "        BlockController.stackBlocks(" + $id1.text + ", " + $id2.text + ");\n"; }
    ;

coordinates returns [String coords]
    :    '(' x=PosInt ',' y=PosInt ')'          { $coords = $x.text + ", " + $y.text; }
    ;

BlockId
    :    ('A'..'Z')+
    ;

PosInt
    :    ('0'..'9') ('0'..'9')* 
    ;

WS  
    :   (' ' | '\t' | '\r'| '\n')               -> channel(HIDDEN)
    ;

(Note that for simplicity this grammar requires semi-colons to separate each instruction.)

(请注意，为了简单起见，该语法要求分号分隔每条指令。)

There are of course other ways to do this sort of thing, but this seems like the simplest to me.

当然还有其他的方法来做这类事情，但对我来说这似乎是最简单的。

Good luck!

好运！

---

Update

更新

So I went ahead and "finished" my original post (fixing a few bugs in the above grammar) and testing it on a simple script.

因此，我继续“完成”了我的原始文章(修复了上面语法中的一些错误)，并在一个简单的脚本上测试它。

Here is the .java file I used to test the above grammar (taken from the code stubs you posted above). Note that in your situation, you would probably want to make the script filename (in my code "script.blockspeak") into a command line parameter. Also, of course the Block and BlockController classes would instead come from your jar.

下面是我用来测试上述语法的.java文件(取自您在上面发布的代码存根)。注意，在您的情况下，您可能希望将脚本文件名(在我的代码“script.blockspeak”中)转换为命令行参数。当然，块和块控制器类也会来自jar。

BlockTest.java:

BlockTest.java:

import org.antlr.v4.runtime.*;

class Block {
    private String name;
    private int xCoord;
    private int yCoord;

    // Other Getters, setters, ctors, etc.
    public Block(int x, int y) { xCoord = x; yCoord = y; }

    public int getXCoord() { return xCoord; }
    public int getYCoord() { return yCoord; }

    public void setXCoord(int x) { xCoord = x; }
    public void setYCoord(int y) { yCoord = y; }

    public void setCoords(int x, int y) {
        setXCoord(x);
        setYCoord(y);
    }
}

class BlockController {
    public static void moveBlock(Block block, int newXCoord, int newYCoord) {
        block.setCoords(newXCoord, newYCoord);
    }

    public static void stackBlocks(Block under, Block onTop) {
        // Stack "onTop" on top of "under".
        // Don't worry about the math here, this is just for an example.
        onTop.setCoords(under.getXCoord() + onTop.getXCoord(), under.getYCoord());
    }
}

public class BlocksTest {
    public static void main(String[] args) throws Exception {
        ANTLRFileStream in = new ANTLRFileStream("script.blockspeak");
        BlockSpeakLexer lexer = new BlockSpeakLexer(in);
        CommonTokenStream tokens = new CommonTokenStream(lexer);
        BlockSpeakParser parser = new BlockSpeakParser(tokens);
        parser.program();
    }
}

And here are the command lines I used (on my MacBook Pro):

以下是我使用的命令行(在我的MacBook Pro上):

> java -jar antlr-4.4-complete.jar BlockSpeak.g
> javac -cp .:antlr-4.4-complete.jar *.java
> java -cp .:antlr-4.4-complete.jar BlocksTest > BlockProgram.java

This was the input script:

这是输入脚本:

script.blockspeak:

script.blockspeak:

block A at (0, 10);                                                                                                                                            
block B at (0, 20);
stack A on B;

And this was the output:

这是输出:

BlockProgram.java:

BlockProgram.java:

//import com.whatever.stuff;

public class BlockProgram {
    public static void main(String[] args) {


        Block A = new Block(0, 10);

        Block B = new Block(0, 20);

        BlockController.stackBlocks(A, B);


    } // main()
} // class BlockProgram

You would of course then have to compile and run BlockProgram.java for each script.

当然，您必须编译并运行BlockProgram。java为每个脚本。

---

In answer to one of the questions in your comment (#3), there are a couple more complex options I first contemplated that might streamline your "user experience".

在回答你的评论中的一个问题(#3)时，我首先考虑了一些更复杂的选项，可以简化你的“用户体验”。

(A) Instead of using the grammar to generate a java program that you then have to compile and run, you could embed the calls to the BlockController directly into the ANTLR actions. Where I created strings and passed them up from one non-terminal to the next, you could have java code there directly doing your Block commands whenever an instruction rule is recognized. This would require a bit more complexity with respect to the ANTLR grammar and imports, but it's technically doable.

(A)您可以将对BlockController的调用直接嵌入到ANTLR操作中，而不是使用语法生成java程序，然后进行编译和运行。当我创建字符串并将它们从一个非终端传递到下一个时，您可以让java代码在那里直接执行您的块命令，只要一条指令规则被识别。对于ANTLR语法和导入，这将需要稍微复杂一点，但从技术上来说是可行的。

(B) If you were to do option A, you could then go a step further and create an interactive interpreter ("shell"), where the user is presented with a prompt and just types in "blockspeak" commands at the prompt, which are then parsed and executed directly, displaying the results back to the user.

(B)如果你选择一个,你可以更进一步,创建一个交互式解释器(“壳”),用户提供的提示和类型在“blockspeak”命令提示符,然后解析和执行直接显示结果返回给用户。

Neither of these options are all that much harder to accomplish in terms of complexity, but they each require doing a lot more coding that would be beyond the scope of a Stack Overflow answer. That's why I opted to present a "simpler" solution here.

从复杂性的角度来看，这两个选项都没有那么难实现，但它们都需要进行更多的编码，而这些编码将超出堆栈溢出答案的范围。这就是为什么我选择在这里提出一个“更简单”的解决方案。

#3

The eval() in ExpParser is implemented through method calls; it's just that the calls have shortcut syntax in the form of operators.

ExpParser中的eval()是通过方法调用实现的;只是调用具有操作符形式的快捷语法。

As an exercise, change ExpParser adding a Calculator class with (unimplemented) methods for mathematical operators, add(), multiply(), divide(), and so on, and then change the rules to use those methods instead of the operators. Thus, you'll understand the basis of what you need to do for your BlockSpeak interpreter.

作为练习，更改ExpParser为数学操作符添加一个计算器类(未实现的)方法，添加()、乘法()、分隔()等等，然后更改规则以使用这些方法而不是操作符。因此，您将理解您需要为您的BlockSpeak解释器做什么。

additionExp returns [double value]
    :    m1=multiplyExp       {$value =  $m1.value;} 
         ( '+' m2=multiplyExp {$value = Calculator.add($value, $m2.value);} 
         | '-' m2=multiplyExp {$value = Calculator.subtract($value, $m2.value);}
         )* 
    ;