ANTLRv4 grammar does not work correctly

[dcamenisch]

When using the ANTLRv4 grammar defined here with antlr4-4.13.1 and the following file (from the example directory) to parse:

grammar Hello;
r   : 'hello' ID;
ID  : [a-z]+ ;
WS  : [ \t\r\n]+ -> skip ;

I get the following error(s):

line 1:8 mismatched input 'Hello' expecting {RULE_REF, TOKEN_REF}

followed by

line 2:0 mismatched input 'r' expecting {<EOF>, OPTIONS, TOKENS, CHANNELS, 'import', 'fragment', 'protected', 'public', 'private', 'mode', AT, RULE_REF, TOKEN_REF}

With the antlr tool I can see that 'Hello' gets identified as ID. Not sure why.

[KvanTTT]

Have you tried the following grammar?

grammar Hello;
r   : 'hello' ID;
HELLO: 'hello';
ID  : [a-z]+ ;
WS  : [ \t\r\n]+ -> skip ;

[dcamenisch]

This does not change anything :/ The problem is that the ANTLRv4 grammar should parse this given example grammar and fails on the grammar type already.

[kaby76]

This grammar works fine using a generated CSharp target driver from template.

02/06-16:06:34 ~/issues/g4-3959/antlr/antlr4
$ trgen -t CSharp
CSharp  LexBasic.g4 success 0.0394926
CSharp  ANTLRv4Parser.g4 success 0.010825
CSharp  ANTLRv4Lexer.g4 success 0.0068014
Rendering template file from CSharp/Other.csproj to Generated-CSharp/Other.csproj
Rendering template file from CSharp/st.build.ps1 to Generated-CSharp/st.build.ps1
Rendering template file from CSharp/st.build.sh to Generated-CSharp/st.build.sh
Rendering template file from CSharp/st.clean.ps1 to Generated-CSharp/st.clean.ps1
Rendering template file from CSharp/st.clean.sh to Generated-CSharp/st.clean.sh
Rendering template file from CSharp/st.Encodings.cs to Generated-CSharp/st.Encodings.cs
Rendering template file from CSharp/st.ErrorListener.cs to Generated-CSharp/st.ErrorListener.cs
Rendering template file from CSharp/st.makefile to Generated-CSharp/st.makefile
Rendering template file from CSharp/st.perf.sh to Generated-CSharp/st.perf.sh
Rendering template file from CSharp/st.run.ps1 to Generated-CSharp/st.run.ps1
Rendering template file from CSharp/st.run.sh to Generated-CSharp/st.run.sh
Rendering template file from CSharp/st.test-cover.sh to Generated-CSharp/st.test-cover.sh
Rendering template file from CSharp/st.Test.cs to Generated-CSharp/st.Test.cs
Rendering template file from CSharp/st.test.ps1 to Generated-CSharp/st.test.ps1
Rendering template file from CSharp/st.test.sh to Generated-CSharp/st.test.sh
Rendering template file from CSharp/Test.csproj.st to Generated-CSharp/Test.csproj.st
Copying source file from C:/msys64/home/Kenne/issues/g4-3959/antlr/antlr4/readme.md to Generated-CSharp/readme.md
Copying source file from C:/msys64/home/Kenne/issues/g4-3959/antlr/antlr4/LexBasic.g4 to Generated-CSharp/LexBasic.g4
Copying source file from C:/msys64/home/Kenne/issues/g4-3959/antlr/antlr4/in to Generated-CSharp/in
Copying source file from C:/msys64/home/Kenne/issues/g4-3959/antlr/antlr4/desc.xml to Generated-CSharp/desc.xml
Copying source file from C:/msys64/home/Kenne/issues/g4-3959/antlr/antlr4/ANTLRv4Parser.g4 to Generated-CSharp/ANTLRv4Parser.g4
Copying source file from C:/msys64/home/Kenne/issues/g4-3959/antlr/antlr4/ANTLRv4Lexer.g4 to Generated-CSharp/ANTLRv4Lexer.g4
Copying source file from C:/msys64/home/Kenne/issues/g4-3959/antlr/antlr4/CSharp/LexerAdaptor.cs to Generated-CSharp/LexerAdaptor.cs
02/06-16:06:41 ~/issues/g4-3959/antlr/antlr4
$ cd Generated-CSharp/
02/06-16:06:44 ~/issues/g4-3959/antlr/antlr4/Generated-CSharp
$ make
bash build.sh
  Determining projects to restore...
  Restored C:\msys64\home\Kenne\issues\g4-3959\antlr\antlr4\Generated-CSharp\Test.csproj (in 315 ms).
MSBuild version 17.8.3+195e7f5a3 for .NET
  Determining projects to restore...
  All projects are up-to-date for restore.
C:\msys64\home\Kenne\issues\g4-3959\antlr\antlr4\Generated-CSharp\LexerAdaptor.cs(76,18): warning CS0414: The field 'LexerAdaptor._i
nsideOptionsBlock' is assigned but its value is never used [C:\msys64\home\Kenne\issues\g4-3959\antlr\antlr4\Generated-CSharp\Test.c
sproj]
  Test -> C:\msys64\home\Kenne\issues\g4-3959\antlr\antlr4\Generated-CSharp\bin\Debug\net7.0\Test.dll

Build succeeded.

C:\msys64\home\Kenne\issues\g4-3959\antlr\antlr4\Generated-CSharp\LexerAdaptor.cs(76,18): warning CS0414: The field 'LexerAdaptor._i
nsideOptionsBlock' is assigned but its value is never used [C:\msys64\home\Kenne\issues\g4-3959\antlr\antlr4\Generated-CSharp\Test.c
sproj]
    1 Warning(s)
    0 Error(s)

Time Elapsed 00:00:04.98
02/06-16:06:51 ~/issues/g4-3959/antlr/antlr4/Generated-CSharp
$ cat ../in
grammar Hello;
r   : 'hello' ID;
ID  : [a-z]+ ;
WS  : [ \t\r\n]+ -> skip ;02/06-16:07:02 ~/issues/g4-3959/antlr/antlr4/Generated-CSharp
$ cat ../in | ./bin/Debug/net7.0/Test.exe
CSharp 0 stdin success 0.0505645
02/06-16:07:16 ~/issues/g4-3959/antlr/antlr4/Generated-CSharp
$ echo $?
0
02/06-16:07:20 ~/issues/g4-3959/antlr/antlr4/Generated-CSharp
$

It also works for a generated Java target driver.

$ cat ../in | bash run.sh
Java 0 stdin success 0.015
02/06-16:09:18 ~/issues/g4-3959/antlr/antlr4/Generated-Java
$

And, it also works using "testrig" (see https://github.com/kaby76/g4-scripts/blob/main/testrig.sh)

$ bash /c/Users/Kenne/Documents/GitHub/g4-scripts/testrig.sh --tree in
Grammar ANTLRv4
Start grammarSpec
Antlr C:\Users\Kenne/.m2/repository/org/antlr/antlr4/4.13.1/antlr4-4.13.1-complete.jar
(grammarSpec (grammarDecl (grammarType grammar) (identifier Hello) ;) (rules (ruleSpec (parserRuleSpec r : (ruleBlock (ruleAltList (labeledAlt (alternative (element (atom (terminalDef 'hello'))) (element (atom (terminalDef ID))))))) ; exceptionGroup)) (ruleSpec (lexerRuleSpec ID : (lexerRuleBlock (lexerAltList (lexerAlt (lexerElements (lexerElement (lexerAtom [a-z]) (ebnfSuffix +)))))) ;)) (ruleSpec (lexerRuleSpec WS : (lexerRuleBlock (lexerAltList (lexerAlt (lexerElements (lexerElement (lexerAtom [ \t\r\n]) (ebnfSuffix +))) (lexerCommands -> (lexerCommand (lexerCommandName (identifier skip))))))) ;))) <EOF>)
02/06-16:12:25 ~/issues/g4-3959/antlr/antlr4
$

You did not say how you test the grammar. We need more information.