Move some options inside the grammar file

[curvedmark]

Currently, some CLI options are critical to the action code. Having to specify them on a command line (or inside a Makefile) means scattering of the code:

• --export-var

  If action needs to recursively called the parser, then this API is critical to it.

• --allowed-start-rule

  It's very unlikely authors will frequently change this option without modifying the grammar

So I propose move these options inside grammar. To bikeshed, JSON might be used:

{
    "export-var": "var parser",
    "allowed-start-rule": ["start", "non-start"]
}
{
    // intializer
}

start = 'a'

nonstart = 'b'

The corresponding CLI options could still be perserved, to override options in the grammar file.

[curvedmark]

just a side note: "export-var": "var parser" looks somewhat awkward. Maybe pegjs could automatically add var to it if there is no dot inside the variable name?

[curvedmark]

actually, this is a rather different beast. authors might do "export-var": "var parser = module.exports", in which case it should probably be "export-var": ["parser", "module.exports"]. maybe I should open a new issue for it, since this also applies to the CLI.

[curvedmark]

or overloading options:

{
    // intializer

    // pegjs will respect these
    options.exportVar = ["parser", "module.exports"]
    options.allowedStartRules = ["start", "non-start"]
}

but this probably means options specified in CLI are defaults, not to override anymore.

bikeshedding: maybe rename exportVar to exports, allowedStartRules to startRules

[dmajda]

If action needs to recursively called the parser, then this API is critical to it.

You have such a use case? If so, can you describe it?

---

I'll have a look at the possibility to specify options inside the grammar after 1.0. There are more pressing things to deal with now.

As for the other ideas expressed in the comments, please file separate issues or use the PEG.js Google Group if you think they deserve discussion in a wider context.

[curvedmark]

You have such a use case? If so, can you describe it?

If the language allows including another files, you might want to load the file and recursively call parser in the action.

But I agree this is a pretty weak argument, for two reasons. One would be that you could just generate a tree, and walk that tree to recursively call the parser. Another would be that loading files might be asynchronous, and I guess you probably don't want to complicate the parser to allow for asynchronous actions.

In my case, the language is pretty simple (it's a preprocess syntax for another language) that I don't want to generate a tree for it, and I can synchronously load files. So I could just put preprocessing inside actions. But in reality, I end up using plain regular expressions to parse directives, and this use case doesn't concern me any more.

If you think this use case is too trivial to justify it. I understand.

I'll have a look at the possibility to specify options inside the grammar after 1.0. There are more pressing things to deal with now.

Feel free to do so, and I'm grateful you spend time improving pegjs. :)

[curvedmark]

And the issue regarding --export-var: #140

[dmajda]

There is some more interest in this topic, so I'll describe my current thinking.

Syntax

I'd like to have an options block at the very beginning of the grammar. It would contain the options:

options {
  ...
}

The initializer block, which is now an unnamed block syntactically, should probably be also named for consistency:

initialize {
  ...
}

The most important unresolved question I have is: In which format should the options be?

One possibility is to use JSON:

options {
  "allowedStartRules": ["start", "non-start"]
}

This is very generic and most authors would know the syntax. On the other hand, I can imagine something more simple syntactically:

options {
  allowedStartRules: start, non-start
}

This could be implemented in several ways.

One of them would be to just allow only known options and make their values syntactically similar (or even exactly the same) to their CLI versions. The obvious disadvantage is loss of generality (which is useful e.g. for plugins).

Another way would be to use NEON (or something similar), so there is both simple syntax and generality at the same time.

Semantics

The built-in default values of the options would be overriden by the ones specified in the grammar, which would be further overridden by the ones passed to PEG.buildParser or the pegjs binary. I think this sequence makes most sense. I also think it doesn't make sense to think too deeply about which options should/shouldn't be overridable and where (based on how much sense that makes for each option). It would just complicate things.

[andreineculau]

Continuing the discussion of #250 , and now having read this issue's thread, I see things differently. Specifically, specifying a start rule of a grammar's parser is not part of the grammar itself, and that's why I went for the Emacs' style of options as comments in my PR. We had these diff views also in #178 (comment) and given that meanwhile I have reused the same grammars (and updated grammars independently) with different outputs (associated actions), I see only benefits (of not coupling everything).

Note: I admit I'm biased (but that could be good, in this context) - core-pegjs concatenates grammars for reuse. Finding and removing EmacsFileVars of "subgrammars" is trivial. Alternatively, with the above proposal, I'd have to fork PEGjs even more and say allow an options {...} block more than once, but ignore anything but the first occurrence.

On the subject of your current thinking:

• Wouldn't it be confusing to have an options {...} block which will have a data structure inside, and an initializer {...} block with JS code inside? I know I would find it so, especially if not going with a JSON syntax. There aren't many options to begin with, and I think that's a good thing (KISS), so I don't share the focus on visual brevity, at the expense of simplicity/uniformity.
• What's wrong with

options {
  return {
    allowedStartRules: ["start", "non-start"]
  }
}

• You mentioned briefly plugins, in a different context, but just to make sure: you're not thinking that a grammar would have an options {...} block specifying options for plugin X (that it might actually require, in order for the parser to work properly), right?

[StoneCypher]

Notably, the example option given here is 95% of what you need for minimalist Typescript support #562 ; this gives you enough to do es6 modules if you just don't impose it as an assignment, and then add a string for the return type of parse and you're done