New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Import/include other grammars #38
Comments
I agree that this is an important feature, I want to do this after version 1.0. (BTW I don't like the Python-like syntax you propose — something similar to Node.js's |
Would you consider it for inclusion before 1.0 if provided with a patch ? I agree on your remark about the python syntax. |
+1 for this feature |
@ceymard Yes, I would consider it. |
+1 for the feature and +1 for |
@dmajda @ceymard Do you have any thoughts already on how to implement this? I need this for a project at work and will try to implement. The question is should this be just an addition to split grammars into multiple files or something like inheritance, so one could inherit all rules for example and then overwrite specific rules in the new grammar. |
@dignifiedquire I am currently thinking about syntax & semantics that can probably be best explained by an example: static-languages.pegjs
dynamic-languages.pegjs
all-languages.pegjs
Each This design is deliberately similar to Node.js. Using namespaces will avoid conflicts. There are two downsides I see:
|
@dmajda As the
The Also if you want to use namespaces, do you think there should be only one namespace per file or should there be a way of creating multiple namespaces in one file like this:
|
I'm not much of a fan of I'd also like to keep things simple and define namespaces implicitly only by requiring files. I don't see a big need for anything more complicated. |
How about simply:
Colons are a compromise, but they are used to separate namespaces in many places: C++, C#, XML, etc. |
|
|
I was thinking of a quick workaround on this topic - to solve simple inheritance only - glue pegjs files together, while having everything namespaced. This might make grammars too verbose, and involves a building step - but looking at the bright side, it would force you to have granular DRY&OTW grammars And regarding the markup, no saying that this is a proper fit to this thread, but just an option to consider, I was going for a simple __
|
@andreineculau I'm basically already doing this with a build step, so if you and others are just looking for something to generate useful parsers from a grammar with a dependency tree (where a single parser implementing the combined grammar is generated), I might clean what I have up and release it so the discussion can refocus on how to deal with this in a more permanent way. Another thing: approaching this primarily by designing extensions to the grammar syntax misses something important, which is that one of the main reasons we all have the itch to pull in rules from other grammars (another being clarity) is the need to write parsers that share a lot of logic. So, while generated parsers might never be meaningfully re-composable at parse-time, it seems important that a tree of grammars generate a tree of parsers, rather than one monolithic parser. It's most important when a set of parsers will be part of a web UI, but it generally doesn't hurt to avoid unnecessary bloat in generated code. |
@odonnell +1 for releasing anything - no matter if you have the time to clean it up and +1 for the clarification. This should be treated as a quick workaround, not a long-term proper solution. |
@odonnell my take on it is online at https://github.com/andreineculau/core-pegjs - please poke me if you have something better. |
+1 for this feature |
👍 |
2 similar comments
👍 |
👍 |
I went and wrote a plugin/extension for PEG.js that does imports: https://github.com/casetext/pegjs-import. |
+1 for this as well. |
I implement this in #308 in generic way: inclusion of grammar is only one way to implement decomposition rules. |
… grammar and compiler. Import clause syntax: ``` @<alias> = <string with path to .pegjs file> ``` Import clauses expected before initializer code block. Usage of imported rules: - @<alias> -- use default parse rule (implementation-defined, for example, first in imported grammar). - @<alias>:<rule> -- use specified parse rule. Implementation: - All import clauses appears in AST `grammar` node in `imports` node property. This property contains array of AST `import` nodes with properties `alias` and `path`. - All AST `rule_ref` nodes now have `namespace` property with alias of imported grammar or `null` if rule defined in the grammar itself. Also, if `name` property is `null` and `namespace` property is not, then will be used default rule of imported grammar. - Support in compiler -- skip some checks for references to imported rules.
… grammar and compiler. Import clause syntax: ``` @<alias> = <string with path to .pegjs file> ``` Import clauses expected before initializer code block. Usage of imported rules: - @<alias> -- use default parse rule (implementation-defined, for example, first in imported grammar). - @<alias>:<rule> -- use specified parse rule. Implementation: - All import clauses appears in AST `grammar` node in `imports` node property. This property contains array of AST `import` nodes with properties `alias` and `path`. - All AST `rule_ref` nodes now have `namespace` property with alias of imported grammar or `null` if rule defined in the grammar itself. Also, if `name` property is `null` and `namespace` property is not, then will be used default rule of imported grammar. - Support in compiler -- skip some checks for references to imported rules.
@dmajda I'm coming late to this party, but I wonder how often we need to import many rules from another library. I would love to be able to import things like import { SchemalessUrl, Url } from "./Urls.pegjs"
Token
= PhoneNumber
/ Url
/ SchemalessUrl Namespacing has been an issue for me as I try and explore writing otherwise-composable grammars. I'm stuck right now including files in files and naming things the way PHP functions were named before they introduced proper namespaces: |
I wouldn't mind being able to split grammars between files, simply for organization and composition. It would make them easier to test and re-use, as well as providing a way swap grammars dynamically, maybe? Just some thoughts. The JavaScript example that I used as a base is over 1,300 lines. It took a while to learn where everything was, and jump around and edit different sections. |
@mikeaustin I see this feature as some kind of Node.JS
|
I agree, splitting grammars and making them modular is a great feature, however handling these case's would be a a problem: IMO, a temporally convenient approach would be creating a new addon for PEG.js (independent from PEG.js) that defines a keyword for importing (for example @load(anotherGrammarFileLocation) ) keyword should not part of javacsript/peg.js grammar, Example:integers.pegjs
main.pegjs
Note using this method, if someone did not define the start grammar, and placed @load before "arrayOfInteger" peg.js will assume the first grammar as the start ( integers grammar) One approach to handle this is , use same names of filename and start grammar and let the new ad-don manually configure the start attribute as the file name, or substitute all content at the end of file. user should be responsible of any duplication . |
I just want to highlight that this issue is primarily an optimization request, because composability/modularity is something that you can achieve on your own, especially when you control the full spectrum of the grammar. If you're not comfortable with a grammar 1k-lines long, then split it up, and concatenate it back as you see fit before pumping it into pegjs. |
… grammar and compiler. Import clause syntax: ``` @<alias> = <string with path to .pegjs file> ``` Import clauses expected before initializer code block. Usage of imported rules: - @<alias> -- use default parse rule (implementation-defined, for example, first in imported grammar). - @<alias>:<rule> -- use specified parse rule. Implementation: - All import clauses appears in AST `grammar` node in `imports` node property. This property contains array of AST `import` nodes with properties `alias` and `path`. - All AST `rule_ref` nodes now have `namespace` property with alias of imported grammar or `null` if rule defined in the grammar itself. Also, if `name` property is `null` and `namespace` property is not, then will be used default rule of imported grammar. - Support in compiler -- skip some checks for references to imported rules.
… grammar and compiler. Import clause syntax: ``` @<alias> = <string with path to .pegjs file> ``` Import clauses expected before initializer code block. Usage of imported rules: - @<alias> -- use default parse rule (implementation-defined, for example, first in imported grammar). - @<alias>:<rule> -- use specified parse rule. Implementation: - All import clauses appears in AST `grammar` node in `imports` node property. This property contains array of AST `import` nodes with properties `alias` and `path`. - All AST `rule_ref` nodes now have `namespace` property with alias of imported grammar or `null` if rule defined in the grammar itself. Also, if `name` property is `null` and `namespace` property is not, then will be used default rule of imported grammar. - Support in compiler -- skip some checks for references to imported rules.
… grammar and compiler. Import clause syntax: ``` @<alias> = <string with path to .pegjs file> ``` Import clauses expected before initializer code block. Usage of imported rules: - @<alias> -- use default parse rule (implementation-defined, for example, first in imported grammar). - @<alias>:<rule> -- use specified parse rule. Implementation: - All import clauses appears in AST `grammar` node in `imports` node property. This property contains array of AST `import` nodes with properties `alias` and `path`. - All AST `rule_ref` nodes now have `namespace` property with alias of imported grammar or `null` if rule defined in the grammar itself. Also, if `name` property is `null` and `namespace` property is not, then will be used default rule of imported grammar. - Support in compiler -- skip some checks for references to imported rules.
…mar to another. Conflicts: lib/compiler/index.js package.json
… in the compiler: ``` import { rule1, rule2 as alias2, ..., ruleN } from '<string with path to the .peggy file>'; ``` Import clauses expected before (top) initializer code block. All import clauses appeared in the `grammar` AST node in the `imports` node property. This property contains array of AST `import` nodes with the`rules` and `path` properties. `rules` contains array of the `imported_rule` AST nodes.
… in the compiler: ``` import { rule1, rule2 as alias2, ..., ruleN } from '<string with path to the .peggy file>'; ``` Import clauses expected before (top) initializer code block. All import clauses appeared in the `grammar` AST node in the `imports` node property. This property contains array of AST `import` nodes with the`rules` and `path` properties. `rules` contains array of the `imported_rule` AST nodes.
It could be extremely useful to have the ability to define grammars by importing rules from other grammars.
Several ideas ;
Ideally, this would not re-generate the whole code in every .pegjs that includes another ; maybe we would have to modify a little the behaviour of parse() to something of the like ;
Editing as per what you were saying in the
options
issue ;And at the end, if
startPos != 0 && result !== null
, we don't check if we went untilinput.length
, but instead return the result as well as the endPos (don't really know how to do that elegantly - maybe simply modifying the options parameter ?).It would allow reusability of grammars and modularisation of the code, which I think are two extremely important aspects of coding in general.
The text was updated successfully, but these errors were encountered: