Discussion of future of antlr4ide

[timeraider4u]

I have opened this issue to discuss the future of antlr4ide development. Feel free to provide feedback and comments. Right now there is a lack of development. Antlr4ide has been converted to a community project but still lacks to attract new contributors. Open issues are already open for a while and the code in the master branch would require some major changes (e.g, converting .xtend files to Java classes as described in #162). Also the Xtext framework which is used under the hood is sometimes hard to understand. And I think it is a pitty that a project with ANTLR in its name is not using a "pure" ANTLR grammar for its work. So right now I see three roads to go. I will list them below together with some of their drawbacks that came up to my mind so far:

1. Using the existing code base and continue re-factoring. This way we do not have to re-write anything from scratch but can build on the existing project structure together with its code and its test cases. Drawbacks are that it uses Xtext and Xtend with all their already mentioned disadvantages. Another drawback is that until now I was the only one willing to do the work for converting Xtend to Java. It will take man-hours to complete and will not gain us any additional value. Time that might be spend better elsewhere.
2. Using an Eclipse-only approach as proposed in https://github.com/antlr4ide/antlr4ide-eclipse Personally I like the idea of getting a clean Eclipse plug-in project. Drawbacks are that we might have to re-write test cases and any editor support from scratch. This includes that we might have to fiddle around with Eclipse GUI plug-in extensions.
3. Using LSP4J/LSP4E to implement our own language server for ANTLR4 and an Eclipse plug-in for the client. The language server protocol (LSP for short) seems to be pretty cool. The idea is basically that you implement support for a programming language just once (you write a "server" that can run as its own process) and have multiple IDEs (e.g., Visual Studio, Eclipse, Atom Editor) that just spawn of this binary in a new process, can communicate with it over JSON-RPC and make use of it. LSP4j provides a Java implementation of the LSP standard. LSP4E provides the client text editor support of LSP for Eclipse. So basically we would have one project for the LSP server that is a stand-alone Java project which provides code formatting, completion and validation support, one Eclipse plug-in project which uses the LSP4E extensions, one VSCode plug-in which uses the VSCode extensions for LSP and so on. What I would like especially about this approach is that it reflects the "IDE" in the name of ANTLR4IDE, as any IDE that supports LSP can also easily support ANTLR4. Drawbacks are that we would have to use an additional framework and that LSP currently lacks support for syntax highlighting (but it is in the backlog for the next version, as described in Proposal for (Semantic) Coloring (see #18) microsoft/language-server-protocol#124). So maybe we would have to wait some time for the pull request to get integrated... Advantages are that LSP is getting support by Microsoft, Redhat and others. LSP4E is still an incubator project but it has already been integrated into Eclipse Che and Eclipse Oxygen.

[HSorensen]

For 1) or 3) I am not sure this is much different from what we have now.

For 2) I have made some progress on my branch
https://github.com/HSorensen/antlr4ide-eclipse

The current version is using an antlr generated grammar to retrieve the tokens.
github.com/antlr/grammars-v4/tree/master/antlrv4

There is no optimization right now so the full grammar is scanned by each keystroke (cough.. cough..).

The syntax highlighting and outlineview works, but properties, folding, tool generation has not been implemented. I should warn that the highlighting is setup for effect and will not win any beauty contest.

That said, I really like this approach to Eclipse Editor. In fact, this editor is trivially extended to any antlr grammar.

[HSorensen]

One thing that is really nice about using an implementation closer to native eclipse API it is relatively easy to enhance the editor with specific content from the grammars.
E.g. to add Lexer Modes to the outline view, is done in ~30 lines of code. see commit "baeba6 Add lexer modes to outlineview"

[HSorensen]

In the discussions regarding java vs xtend I am still trying to figuring out what xtend provides over using the Eclipse API directly from Java.

Take the AntlrConsole implementation as an illustration.

In Java:

package org.github.antlr4ide.console;

import org.eclipse.ui.console.ConsolePlugin;
import org.eclipse.ui.console.IConsole;
import org.eclipse.ui.console.IConsoleFactory;
import org.eclipse.ui.console.IOConsole;

public class AntlrConsoleFactory implements IConsoleFactory {
	public final static String ANTLR_CONSOLE = "Antlr Console"; 
	public final static String ANTLR_CONSOLE_IMAGE = "console.png";

	@Override
	public void openConsole() {
	    // check if Antlr Console is already added:
	    IOConsole antlrConsole = getConsole(); 
	    ConsolePlugin.getDefault().getConsoleManager().showConsoleView( antlrConsole );
	}

	public IOConsole getConsole() {
	    for(IConsole console:ConsolePlugin.getDefault().getConsoleManager().getConsoles()) {
	    	if(console.getName().equals(ANTLR_CONSOLE)) { return (IOConsole) console; }
	    }
	    // If not found add Antlr Console to list
    	IOConsole antlrConsole = new IOConsole(ANTLR_CONSOLE, ConsolePlugin.getImageDescriptor(ANTLR_CONSOLE_IMAGE));
    	ConsolePlugin.getDefault().getConsoleManager().addConsoles( new IConsole[] { antlrConsole } );
	    return antlrConsole;
	}
}

Same in xtend:

package com.github.jknack.antlr4ide.ui.console

import org.eclipse.ui.console.IConsoleFactory
import org.eclipse.ui.console.IOConsole
import org.eclipse.ui.console.ConsolePlugin
import com.google.inject.Inject
import org.eclipse.xtext.ui.IImageHelper.IImageDescriptorHelper

class AntlrConsoleFactory implements IConsoleFactory {

  public static val ANTLR_CONSOLE = "ANTLR Console"

  @Inject
  static IImageDescriptorHelper imageHelper

  override openConsole() {
    val manager = ConsolePlugin.^default.consoleManager
    manager.showConsoleView(console)
  }

  def static getConsole() {
    val manager = ConsolePlugin.^default.consoleManager
    val existing = manager.consoles

    for (console : existing) {
      if (ANTLR_CONSOLE == console.name)
        return console as IOConsole
    }

    val console = new IOConsole(ANTLR_CONSOLE, imageHelper.getImageDescriptor("console.png"))
    manager.addConsoles(#[console])
    return console
  }
}

For this case I am not seeing the benefit of using xtend.

[timeraider4u]

Yeah, your Java Code is more readable than the Xtend one. Also the Java imports are simple and clear. And you are not locked into a certain framework (except for the Eclipse API, which does not really count here ;-)).

[HSorensen]

This is moving forward please join the fun