Medium-term roadmap

[RDambrosio016]

This issue is meant to serve as a roadmap for the next few months to coordinate and discuss what should be done first, as well as give some updates on things i am working on.

First off, the things i am actively working on:

• Scope analysis, most of the code is done, i am fixing many bugs in it (it includes a var ref analyzer and complex hoisting/shadowing logic so its not the easiest thing to debug)
• Autofix, the fixer interface is done, most of the application logic is done, existing rules that need autofix are not done, this depends on incremental reparsing
• Incremental reparsing/relinting, it mostly works, i am however waiting to merge it to add tests and check for any bugs.

Another person also mentioned they made an LSP and a VSC extension for rslint on the rust discord to me, so that is exciting and i will be awaiting the PR for that.

The issue when it comes to deciding what to work on is the fact that i would like to hold off on making more rules until we have all the features that rules should have, which includes autofix. This is vital to the long term of the project, if we rush rules then we end up with buggy rules and a lot of work to do once we add more features for rules. It seems to me that deno_lint ended up suffering from this issue greatly, since they added a lot of rules really fast, which caused bugs to not be caught, see denoland/deno_lint#330, and now they must add documentation, options, and suggestions to every rule. This is why i would rather hold off on adding more.

I am undecided when it comes to order of features, i believe parser tests should be done relatively soon, since the parser is the heart of the linter.

Here is a list of features i would like to get done in the next few months:

• Test262 testing for rslint_parser and 90% compliance (i don't expect full compliance to be achieved soon, there are many weird productions not allowed in ecma2021) 100% seems achievable
• Scope analysis (the driver for rules such as no-undef, no-unused, no-shadow, etc)
• Autofix (Feat: Add autofix #45)
• Incremental relinting
• RegEx parser (this will likely just be put into rslint_parser, ive experimented with this and it involves some weird sub-token logic which will be interesting)
• Code Flow analysis (for rules such as no-unreachable)
• basic LSP/VSC extension, this should be pretty simple and as i mentioned, someone might pr it soon 👀, then it just has to be adapted for incremental relinting
• TypeScript parsing (unlikely to be done in the next few months)

[RDambrosio016]

@Stupremee On the topic of test262, thinking more about it, i would rather make a tester crate which is run by CI to generate a coverage report, having it run each thing as a test is unrealistic and will make CI always fail for the near future.

[Stupremee]

Yep that's the problem I wanted to avoid. As soon as we have 100% coverage we can run it in CI.

[RDambrosio016]

Yeah then we can simply have rslint_parser call the tester and expect 100%

[RDambrosio016]

I think we should also put all of the crates in a crates dir, since the amount has gotten pretty large and rslint_scope will be added soon enough

[RDambrosio016]

Actually, i dont think a new crate is needed, we should just add a coverage command to xtask.

[Stupremee]

I can add test262 support in the next few days.

[RDambrosio016]

That would be great!
You could generate a table of info using the old rslint runner's tablegen, found here
I would love to have a diff between commits but i don't really know if that is possible.

[magurotuna]

Just out of curiosity, why is TypeScript parsing unlikely? Is it technically difficult or something?
Considering the popularity of TypeScript is growing remarkably, it would be better to have support for TypeScript.

[RDambrosio016]

I do not mean typescript wont be supported, it 100% will be, i just meant its unlikely to be done in the next few months because its a lot of parser work and testing and essential analyzers and good js support must come first

[magurotuna]

Ah I see, that sounds great.

[RDambrosio016]

The basic rslint lsp has been merged, ill count that as complete, its not production ready but its great for debugging the linter and we will improve it later on 🙂

[Stupremee]

Do you think it's a good moment to move all crates to a crates folder? There will also be another errors crate soon.

[RDambrosio016]

Haha yeah, its gotten a bit big, xtask will need a couple of changes

[RDambrosio016]

I ran test262 using the parser, and it appears the results are better than i initially thought they would be, it only ran 18k tests for some odd reason, ill have to figure out why, but here are the initial results after fixing a couple of glaring issues:

╒═══════════╤════════╤════════╤════════╤══════════╕
│ Tests ran │ Passed │ Failed │ Panics │ Coverage │
╞═══════════╪════════╪════════╪════════╪══════════╡
│ 17549     │ 16633  │ 347    │ 34     │ 95%      │
└───────────┴────────┴────────┴────────┴──────────┘

There is a very weird bug which occurs sometimes when a comment has a unicode char followed by a period which arises in the LosslessSyntaxTree. I am not quite sure what causes it yet since it is very obscure:

thread '"xtask/src/coverage/test262/test\\built-ins\\Promise\\resolve\\resolve-from-promise-capability.js"' panicked at 'byte index 470 is not a char boundary; it is inside '»' (bytes 469..471) of `"use strict";
// Copyright (C) 2015 André Bargull. All rights reserved.
// This code is governed by the BSD license found in the LICENSE file.

/*---
es6id: 25.4.4.5
description: >
  Resolve function is called after Promise constructor returns.
inf`[...]', crates\rslint_parser\src\lossless_tree_sink.rs:74:28

There are also a couple more glaring issues:

• Template literals which are just two backticks panic because they enter unreachable code in the template handler:

thread '"xtask/src/coverage/test262/test\\language\\expressions\\assignmenttargettype\\direct-callexpression-templateliteral.js"' panicked at 'internal error: entered unreachable code: Anything not template chunk or dollarcurly should have been eaten by the lexer, but EQ was found', crates\rslint_parser\src\syntax\expr.rs:922:18

• switch { .. } often results in infinite recursion, this can be fixed in Curly brackets where parameters should be should be parsed as block statements not object expressions #24 but it should not infinitely recurse in the first place and that is an object literal error recovery issue

• switch(0) { case 1: export default null; default: } infinitely recurses

try{}
catch(){
finally{}

infinitely recurses

Edit: down to ~150 errors woohoo

[RDambrosio016]

The more that i think about it, the more is start to think TS support should come first, it would open up the linter for a ton of users, and we would not be left with a huge burden of integrating TS into some analyzers. Therefore i think i will actively focus on both sorting out parser issues with test262 as well as TS support, that way once the parser is mostly done we can focus on bigger pieces of linting like scope analysis, autofix, regex, etc. If you would like to work on autofix mention it though since i have a lot of already made code for scope analysis and autofix 🙂

[RDambrosio016]

Update: I have now merged the TypeScript PR, i am sure there might be some bugs here and there but it passes the entire test suite taken from swc. For now i will be focusing mostly on polishing the lsp, getting it to work with config and such. Then i'll release 0.3 with everything 🎉

[Aeolun]

Thank you for typescript support! I was going to try and build my own native linter, but it seems someone already had the same idea.