Strange restriction on newlines in macro definitions

[dscorbett]

A newline is only allowed after the equals sign in a macro definition if the next token is not an identifier. I find this unintuitive.

$ cat <<EOF >test1.flex
%%
x =
    "x"
%%
{x} {}
EOF
$ cat <<EOF >test2.flex
%%
x0 = x
x =
    {x0}
%%
{x} {}
EOF
$ cat <<EOF >test3.flex
%%
x =
    x
%%
{x} {}
EOF
$ jflex -q test1.flex
$ jflex -q test2.flex
$ jflex -q test3.flex

Error in file "test3.flex" (line 2): 
Syntax error.
x =
   ^

Error in file "test3.flex" (line 5): 
Macro has not been declared.
{x} {}
^
2 errors, 0 warnings.

[lsf37]

Sorry, for some reason I only saw this one now.

The reason for this restriction is that an identifier can be the start of the next macro definition, but non-identifiers can't be. You'd have to look ahead several tokens to figure out what it is (x happens to be only one token here, but if it was abcd, these would be 4 tokens for a regexp, but only 1 token if it was the left-hand side of a macro definition, i.e. the lexer has to make this decision, it's too late in the parser).

With the more flexible lookahead expressions JFlex has now (this was all written long before that), it might be possible to do that these days, but it would at least require a deeper investigation.

[lsf37]

I'll keep this open until I've figure out how hard it would be to lift the restriction, i.e. whether it can be done with reasonable effort.