Vaxscan #4049

whcox603 · 2024-04-15T01:51:23Z

This is an ANTLR4 parser for the VAX SCAN programming language. See the README.md for a brief description of the language and its history.

I have included 22 small examples that were included with the shipping software kit.

The parser was run through the antlr4-formatter.

teverett · 2024-04-15T03:58:06Z

@whcox603 . I have 4 vaxen. This is a amazing!

kaby76 · 2024-04-15T12:55:08Z

(1) You need a desc.xml so the grammar can be tested. Try this:

<?xml version="1.0" encoding="UTF-8" ?>
<desc xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="../_scripts/desc.xsd">
   <antlr-version>^4.10</antlr-version>
   <targets>Antlr4ng;CSharp;Cpp;Dart;Go;Java;JavaScript;PHP;Python3;TypeScript</targets>
   <inputs>examples</inputs>
</desc>

(2) Please reformat you grammar using antlr-format so that it adheres to the coding standard.

(3) You will need to rename "alt" to "alt_". It does not work for Cpp. The easiest is to just use trrename (trparse vaxscan.g4 | trrename 'alt,alt_' | trsponge -c). Or, you can edit it manually.

(4) The test cases cover 92% of the grammar. Excellent.

…o vaxscan

whcox603 · 2024-04-15T14:00:48Z

I added the desc.xml and fixed the 'alt' issue.

I did format using the Antlr4Formater. I did not realize there were multiple formatters. Is antlr-format the official formatter? I have installed it on my machine, but seem to have some issues running it. I'll work on that today.

Do you have "coverage map" that could share with me? I would be happy to look into adding a few more tests to fill in the holes.

kaby76 · 2024-04-15T17:45:30Z

I did format using the Antlr4Formater. I did not realize there were multiple formatters. Is antlr-format the official formatter? I have installed it on my machine, but seem to have some issues running it. I'll work on that today.

The grammars-v4 Wiki has a section on the formatting. https://github.com/antlr/grammars-v4/wiki#is-there-a-coding-standard-for-antlr4-grammars . The main reason antlr-format is used is because it is configurable, while the Antlr4Formatter is not. The antlr-format package is in TypeScript, so it is integrated in Mike's Antlr4 VS Code extension. I am sure there are other formatters as well.

Do you have "coverage map" that could share with me? I would be happy to look into adding a few more tests to fill in the holes.

The Trash toolkit implements the code coverage with a heat map overlay of the grammar used per input. See trcover.

kaby76 · 2024-04-16T19:58:58Z

Note, if you are trying to be complete in listing a lexer rule for all string literals in a parser rule (i.e., we could rewrite exp4 : exp5 | exp4 '&' exp5 ; => exp4 : exp5 | exp4 AMPERSAND exp5 ; because we have a lexer rule AMPERSAND : '&' ;), then you should know that you are missing lexer rules for '/', '..', '.=', '...', '[', ']', ',', '*', '==', '+''<>', '><', '<', '<=', '>=', '=>', '>', '->'. I may have missed some. I'm not sure why some string literals have lexer rules, while others are missing.

whcox603 · 2024-04-16T20:37:04Z

Let me go back and take a look.

whcox603 · 2024-04-16T21:15:52Z

It appears that I don't actually need the Lexer rules for the the string literals as they are never referred to in the grammar itself. I removed them.

I am unable to run the antlr-format tool. It is probably something I am doing wrong with the download/setup. I entered an issue with antlr-format, but have not heard back yet.

Could I impose upon you to run the grammar through antlr-format and email it back to me or attach it to comment?

bill

kaby76 · 2024-04-17T07:34:28Z

Reformatted: vaxscan.g4.txt

whcox603 · 2024-04-17T12:31:30Z

I updated vaxscan.g4. Thank you for the assist.

I will pursue getting antlr-format working in my environment as well the trash tools.

bill

kaby76 · 2024-04-18T22:16:31Z

@teverett @KvanTTT Could someone please approve the workflow for this PR in order to see how it does?

kaby76 · 2024-04-20T00:23:49Z

Sorry, I just updated my "find unused symbols" scripts (lexer and parser) and found that NONE is defined but not used in the grammar.

whcox603 · 2024-04-20T01:18:02Z

I added pom.xml based on copying and modifying an existing one. I installed maven and it runs cleanly.
I'm not sure what the warning about the system modules path, but I am assuming it is something about my environment.

bill

[INFO] Scanning for projects...
[INFO]
[INFO] ---------------------< org.antlr.grammars:vaxscan >---------------------
[INFO] Building VAX SCAN grammar 1.0-SNAPSHOT
[INFO] from pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- clean:3.2.0:clean (default-clean) @ vaxscan ---
[INFO] Deleting /home/bill/ANTLR4/grammars-v4/vaxscan/target
[INFO]
[INFO] --- resources:3.3.0:copy-resources (copy-resources) @ vaxscan ---
[INFO] skip non existing resourceDirectory /home/bill/ANTLR4/grammars-v4/vaxscan/Java
[INFO]
[INFO] --- build-helper:3.2.0:add-source (add-source) @ vaxscan ---
[INFO] Source directory: /home/bill/ANTLR4/grammars-v4/vaxscan/target/generated-sources/copy/org/antlr/grammars/vaxscan added.
[INFO]
[INFO] --- antlr4:4.11.1:antlr4 (default) @ vaxscan ---
[INFO] ANTLR 4: Processing source directory /home/bill/ANTLR4/grammars-v4/vaxscan
[INFO] Processing grammar: vaxscan.g4
[INFO] Processing grammar: foobarfoobar/vaxscan.g4
[INFO]
[INFO] --- resources:3.3.0:resources (default-resources) @ vaxscan ---
[INFO] skip non existing resourceDirectory /home/bill/ANTLR4/grammars-v4/vaxscan/src/main/resources
[INFO]
[INFO] --- compiler:3.11.0:compile (default-compile) @ vaxscan ---
[INFO] Changes detected - recompiling the module! :source
[INFO] Compiling 12 source files with javac [debug target 11] to target/classes
[WARNING] system modules path not set in conjunction with -source 11
[INFO]
[INFO] --- resources:3.3.0:testResources (default-testResources) @ vaxscan ---
[INFO] skip non existing resourceDirectory /home/bill/ANTLR4/grammars-v4/vaxscan/src/test/resources
[INFO]
[INFO] --- compiler:3.11.0:testCompile (default-testCompile) @ vaxscan ---
[INFO] No sources to compile
[INFO]
[INFO] --- surefire:3.2.2:test (default-test) @ vaxscan ---
[INFO] No tests to run.
[INFO]
[INFO] --- antlr4test:1.22:test (default) @ vaxscan ---
[INFO] Evaluating Scenario: Default Scenario
[INFO] Parsing :/home/bill/ANTLR4/grammars-v4/vaxscan/examples/dif.scn
[INFO] Parsing :/home/bill/ANTLR4/grammars-v4/vaxscan/examples/crlf.scn
[INFO] Parsing :/home/bill/ANTLR4/grammars-v4/vaxscan/examples/micro1.scn
[INFO] Parsing :/home/bill/ANTLR4/grammars-v4/vaxscan/examples/comment.scn
[INFO] Parsing :/home/bill/ANTLR4/grammars-v4/vaxscan/examples/convert.scn
[INFO] Parsing :/home/bill/ANTLR4/grammars-v4/vaxscan/examples/hanoi.scn
[INFO] Parsing :/home/bill/ANTLR4/grammars-v4/vaxscan/examples/cref.scn
[INFO] Parsing :/home/bill/ANTLR4/grammars-v4/vaxscan/examples/dcl.scn
[INFO] Parsing :/home/bill/ANTLR4/grammars-v4/vaxscan/examples/replace.scn
[INFO] Parsing :/home/bill/ANTLR4/grammars-v4/vaxscan/examples/record.scn
[INFO] Parsing :/home/bill/ANTLR4/grammars-v4/vaxscan/examples/trap.scn
[INFO] Parsing :/home/bill/ANTLR4/grammars-v4/vaxscan/examples/macro.scn
[INFO] Parsing :/home/bill/ANTLR4/grammars-v4/vaxscan/examples/time.scn
[INFO] Parsing :/home/bill/ANTLR4/grammars-v4/vaxscan/examples/crypt.scn
[INFO] Parsing :/home/bill/ANTLR4/grammars-v4/vaxscan/examples/compress.scn
[INFO] Parsing :/home/bill/ANTLR4/grammars-v4/vaxscan/examples/traceback.scn
[INFO] Parsing :/home/bill/ANTLR4/grammars-v4/vaxscan/examples/global.scn
[INFO] Parsing :/home/bill/ANTLR4/grammars-v4/vaxscan/examples/trees.scn
[INFO] Parsing :/home/bill/ANTLR4/grammars-v4/vaxscan/examples/micro2.scn
[INFO] Parsing :/home/bill/ANTLR4/grammars-v4/vaxscan/examples/mms.scn
[INFO] Parsing :/home/bill/ANTLR4/grammars-v4/vaxscan/examples/caps.scn
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 4.776 s
[INFO] Finished at: 2024-04-19T19:08:27-04:00
[INFO] ------------------------------------------------------------------------

kaby76 · 2024-04-20T12:23:59Z

[WARNING] system modules path not set in conjunction with -source 11

I'm not sure what the warning is about, but it likely doesn't matter as the parser testing still works identically to that for the trgen-generated driver/tester. Anyway, the repo workflow no longer tests using the Maven tester. Testing via this method is a convenience, not necessary, because the tester using an older version of Antlr, and does not test any target other than Java.

Ideally, if you add a pom.xml, you should probably change the "top-level pom.xml" so that "mvn clean test" at the repo root will include testing of vaxscan.

whcox603 · 2024-04-20T12:27:58Z

ok. Thanks. I will explore the trgen-generated driver/tester.

I there anything else I need to do at this point?

bill

kaby76 · 2024-04-20T13:17:53Z

I there anything else I need to do at this point?

I don't think so. But, we need a build of the grammar. @teverett @KvanTTT

kaby76 · 2024-04-22T11:29:51Z

... ]W]e need a build of the grammar. @teverett @KvanTTT

Admins, @teverett @KvanTTT , can we please get a build of this PR?

Admins @teverett or @KvanTTT . This is the third time to ask: we need a build, please. The author of the PR and I have spent a good deal of time on this. It should not go to waste.

whcox603 · 2024-05-01T18:33:27Z

Is there any chance this could be merged this week? Has a build been done? Is there any issue that I should be aware of?

Thanks,
bill

kaby76 · 2024-05-01T19:52:20Z

Is there any chance this could be merged this week? Has a build been done? Is there any issue that I should be aware of?

Thanks, bill

There still has not been a build queued up for this PR. "2 workflows awaiting approval
This workflow requires approval from a maintainer." @teverett @KvanTTT @ericvergnaud @parrt

Can someone add some privileges to me for this repo so I can get this done?

parrt · 2024-05-02T01:47:31Z

I will defer to Tom Everett on this one, but I’m not opposed to you having privileges on the grammar repo Dictation in use. Please excuse homophones, malapropisms, and nonsense.

…

On Wed, May 1, 2024 at 12:52 PM Ken Domino ***@***.***> wrote: Is there any chance this could be merged this week? Has a build been done? Is there any issue that I should be aware of? Thanks, bill There still has not been a build queued up for this PR. "2 workflows awaiting approval This workflow requires approval from a maintainer." @teverett <https://github.com/teverett> @KvanTTT <https://github.com/KvanTTT> @ericvergnaud <https://github.com/ericvergnaud> @parrt <https://github.com/parrt> Can someone add some privileges to me for this repo so I can get this done? — Reply to this email directly, view it on GitHub <#4049 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABLUWI4RJLZC6SBISR2KOLZAFBYVAVCNFSM6AAAAABGGN3IE2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOBZGAYTCMBRGQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

whcox603 · 2024-05-09T15:41:42Z

I'm not sure how to interpret the build results. I do see one System Exception complaining that it can't figure out the name of the vaxscan grammar. And I see some errors about not being able to find the java executable when running tests for the Csharp target, but I don't see any indication what the problem was.

kaby76 · 2024-05-09T19:01:55Z

I'm not sure how to interpret the build results. I do see one System Exception complaining that it can't figure out the name of the vaxscan grammar. And I see some errors about not being able to find the java executable when running tests for the Csharp target, but I don't see any indication what the problem was.

@whcox603 Please "sync fork" in Github then update your branch for your PR. It's caused by a couple of problems with the performance check code, which was fixed in the last week.

whcox603 · 2024-05-09T19:38:03Z

Please forgive my ignorance. I did a sync fork in github for my fork, but I am unsure what else I need to do to "update my branch."

kaby76 · 2024-05-09T20:45:58Z

Please forgive my ignorance. I did a sync fork in github for my fork, but I am unsure what else I need to do to "update my branch."

In Github, navigate to your repo. Then: (1) Select the branch from the pull-down menu on the left hand side. (2) Click on "sync fork".

When you are done, write a comment to @teverett to redo the build (if it doesn't already kick off).

whcox603 · 2024-05-09T21:43:23Z

@teverett I have re-sync'd. It would be greatly appreciated if you could kick off another build.

kaby76 · 2024-05-09T23:26:28Z

Looking into the build...

kaby76 · 2024-05-09T23:48:21Z

Sorry, the branch for the PR is still 29 commits behind. Please check github. Or may I suggest Github Desktop. You need to get the branch completely up to date otherwise it won't build.

whcox603 · 2024-05-10T00:01:42Z

My bad. It now says it is up to date.

bill

kaby76 · 2024-05-10T00:17:03Z

@teverett Please start the workflow for this PR. Ty.

kaby76 · 2024-05-10T22:49:42Z

All worked fine except MacOS and Dart. Hangs on 4th input file. Normally tredog would kill a stalled process but it didn't because the OS crashed. The github mac servers are a pain because they are unreliable. Dart works fine elsewhere.

@teverett I think it's good to go. It'll likely all pass fine on the merge.

teverett · 2024-05-11T03:52:46Z

@whcox603 thanks!

whcox603 · 2024-05-12T19:04:51Z

@teverett @kaby76 Thank you both for the help.

William Cox and others added 2 commits April 14, 2024 21:41

Adding vaxscan ANTLR4 parser

ed37347

Update README.md

e830ff6

William Cox added 2 commits April 15, 2024 09:40

Adding desc.xml file and changed alt to alt_ to avoid conflicts.

26eabb8

Merge branch 'vaxscan' of https://github.com/whcox603/grammars-v4 int…

6a652d0

…o vaxscan

Remove the spurious lexer rules for some of the string literals.

80b639e

Grammar reformatted using antlr-format tool.

73f2e96

whcox603 closed this Apr 17, 2024

whcox603 reopened this Apr 17, 2024

KvanTTT added the new-grammar New grammar issue or pull request label Apr 17, 2024

Adding pom.xml file for maven

c54be0b

teverett added the vaxscan label May 9, 2024

Merge branch 'antlr:master' into vaxscan

b248ae7

teverett merged commit d5c2009 into antlr:master May 11, 2024
34 checks passed

whcox603 deleted the vaxscan branch May 13, 2024 00:42

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Vaxscan #4049

Vaxscan #4049

whcox603 commented Apr 15, 2024

teverett commented Apr 15, 2024

kaby76 commented Apr 15, 2024 •

edited

whcox603 commented Apr 15, 2024

kaby76 commented Apr 15, 2024 •

edited

kaby76 commented Apr 16, 2024

whcox603 commented Apr 16, 2024

whcox603 commented Apr 16, 2024

kaby76 commented Apr 17, 2024

whcox603 commented Apr 17, 2024

kaby76 commented Apr 18, 2024

kaby76 commented Apr 20, 2024

whcox603 commented Apr 20, 2024

kaby76 commented Apr 20, 2024 •

edited

whcox603 commented Apr 20, 2024

kaby76 commented Apr 20, 2024

kaby76 commented Apr 22, 2024 •

edited

whcox603 commented May 1, 2024

kaby76 commented May 1, 2024

parrt commented May 2, 2024 via email

whcox603 commented May 9, 2024

kaby76 commented May 9, 2024

whcox603 commented May 9, 2024 •

edited

kaby76 commented May 9, 2024

whcox603 commented May 9, 2024

kaby76 commented May 9, 2024

kaby76 commented May 9, 2024

whcox603 commented May 10, 2024

kaby76 commented May 10, 2024

kaby76 commented May 10, 2024

teverett commented May 11, 2024

whcox603 commented May 12, 2024

Vaxscan #4049

Vaxscan #4049

Conversation

whcox603 commented Apr 15, 2024

teverett commented Apr 15, 2024

kaby76 commented Apr 15, 2024 • edited

whcox603 commented Apr 15, 2024

kaby76 commented Apr 15, 2024 • edited

kaby76 commented Apr 16, 2024

whcox603 commented Apr 16, 2024

whcox603 commented Apr 16, 2024

kaby76 commented Apr 17, 2024

whcox603 commented Apr 17, 2024

kaby76 commented Apr 18, 2024

kaby76 commented Apr 20, 2024

whcox603 commented Apr 20, 2024

kaby76 commented Apr 20, 2024 • edited

whcox603 commented Apr 20, 2024

kaby76 commented Apr 20, 2024

kaby76 commented Apr 22, 2024 • edited

whcox603 commented May 1, 2024

kaby76 commented May 1, 2024

parrt commented May 2, 2024 via email

whcox603 commented May 9, 2024

kaby76 commented May 9, 2024

whcox603 commented May 9, 2024 • edited

kaby76 commented May 9, 2024

whcox603 commented May 9, 2024

kaby76 commented May 9, 2024

kaby76 commented May 9, 2024

whcox603 commented May 10, 2024

kaby76 commented May 10, 2024

kaby76 commented May 10, 2024

teverett commented May 11, 2024

whcox603 commented May 12, 2024

kaby76 commented Apr 15, 2024 •

edited

kaby76 commented Apr 15, 2024 •

edited

kaby76 commented Apr 20, 2024 •

edited

kaby76 commented Apr 22, 2024 •

edited

whcox603 commented May 9, 2024 •

edited