Discussion: support Regular Expressions in lexers

[iahung3]

You have agreed to add support for Regular Expressions in search in #147. This is about Regular Expressions in lexers. It's currently not supported. If support is added, the pros is Lite XL lexers could be ported more easily. The cons is lexers will become more complex.

[SpartanJ]

Yes, this is going to be implemented too and it will support the Lite-XL language definitions.

[iahung3]

Yes, this is going to be implemented too and it will support the Lite-XL language definitions.

I found most Lite XL lexers don't use regex. Are there performance issues with regex in lexers? If there are, I don't think it's the fault of regex. It's the poor quality of the syntax highlighting engine. It could be regex caused the lexers to be too complex, so they avoided it intentionally.

[SpartanJ]

 I found most Lite XL lexers don't use regex. Are there performance issues with regex in lexers?

They usually are slower and much slower, but they give you some flexibility, so it's something that you'll try to avoid unless there's no other alternative. That's why I skipped regex to begin with, I found I thinks only 2 o 3 languages using them and it was easy to replace the regex in favor of lua patterns for those cases.

 If there are, I don't think it's the fault of regex. It's the poor quality of the syntax highlighting engine. It could be regex caused the lexers to be too complex, so they avoided it intentionally.

It will be a mixture, if you have a lexer based on pattern-matching, the regex engine performance is the most important aspect affecting the final performance. The quality and complexity of the particular pattern will be also important, but it's really hard to measure the performance of each pattern, and each regex engine has each own strong and cons with different kinds of patterns. In order to be able to develop fast patterns you'll need to know very well your regex engine. Anyway, this applies for the lua patterns too, complexity of the pattern affects the highlighting performance. Currently highlighting is REALLY fast on ecode and I plan to keep it that way, that's why I try to be very conscious about any change on the highlighter.