Disable non-greedy restriction on regexes

[liaden]

I don't know if this is sufficient, but it compiles, passes local tests, and when compiled it helps my grammar pass tests that highlighted the case where I wanted the regex to be nongreedy.

Fixes #209

[Marwes]

It doesn't look like you use this flag anywhere in the changes from this PR so I think that this just makes the code ignore the non-greedy option altogether (non-greedy regexes act like greedy ones)?

I don't know exactly how to implement non-greedy regexes for LALRPOP but using non-greedy at the end of a token may be a bit tricky to implement as I think it would need to generate code such that repetitions only consume data if the start of any of the defined tokens would not match after it. Just implementing it for using non-greedy strictly inside of a token might be simpler however.

[liaden]

I took a stab at this because I saw @nikomatsakis comment #209 (comment) in the linked issue.

In my case, I was doing r"'.*?[^']'" to capture everything in between single quotes up to the next single quote that does not have a single quote before it and r"''" to then grab an empty string without having to or operator and specify non capturing groups in the first regex.

In my case though,that didn't give me what I needed to parse what I needed, but non-greedy regexes give a decent bit of extra control.

[Marwes]

Might be that you only disabled some validation then and the actual construction of of the actual token matcher (regex) is done elsewhere and this works fine. I guess if this code only checks overlap then the flag can be ignored here as greediness shouldn't affect overlap (I think). Could you add some tests using non-greedy tokens to check that this works as expected (with non-greedy used at the start, middle and end of a token respectively)?

[nikomatsakis]

Thanks for the PR, we're deferring this to get a short term patch out the door, we may consider it later.