[WIP]Feature/issue 660 #666
Conversation
ghost
assigned 
Codecov Report
@@ Coverage Diff @@
## master #666 +/- ##
==========================================
- Coverage 82.94% 82.93% -0.02%
==========================================
Files 56 56
Lines 7501 7507 +6
==========================================
+ Hits 6222 6226 +4
- Misses 1027 1028 +1
- Partials 252 253 +1
Continue to review full report at Codecov.
|
|
|
||
| for isDigit(l.ch) || l.ch == '_' { | ||
|
|
||
| if isDigit(l.ch) { |
There was a problem hiding this comment.
IMO I think the lexer should preserve the underscores in the token. In terms of responsibility, I don't think a lexer should be responsible for any code transformation. Removing them at this stage seems sort of magical.
IMO the token should be simplified at a later stage, maybe during parsing or eval?
|
Well I think @SD10 has a good point. Keeping those information to parser allows us to provide more sophisticated error message |
|
Here we are already transforming the token into a literal for goby/compiler/parser/data_type_parsing.go Line 12 in c00ad0c Here for Float: goby/compiler/parser/data_type_parsing.go Line 26 in c00ad0c I think this is a good place to do the transformation work. This is just my opinion. I'm a newbie 😊 |
|
It's also worth noting that in the Ruby lexer the underscores are maintained: |
|
I will preserve the underscores in the token and handle it in parser, |
1_2_3=> 1231_2.3_4=> 12.341__2=> Error_1=> Error1_=> ErrorThis closes #660