Syntax highlighting sometimes gets out of sync

[DanTup]

I had this issue raised at Dart-Code/Dart-Code#5082 and although I'd only seen it a few times in the past, I just saw it again today. I was typing in a file, and a bunch of colours got out of sync in my editor.

While my first thought was that it was an issue with the semantic token or textmate info, the "Inspect Tokens and Scopes" command shows in the tooltip that the same scopes are applied to the whole of words that are coloured differently.

Unfortunately I don't have reliable steps to reproduce, it just happens sometimes (it might have started when I saved a file and briefly saw "VS Code is waiting for Code Actions").

Here you can see that VS Code says the foreground color is the same for two positions in this word, yet one part is pink and one part is blue:

colours.mp4

[hediet]

This is concerning. Since when did you see this? The last big change in tokenization is a year ago.

Is it correct that this only happens with semantic colorization?

[DanTup]

@hediet I've seen it before and not too recently, so it may have been around for a while. I think I previously thought it was a Dart bug and because I couldn't repro I hadn't dug into it. It was only when it occured yesterday (shortly after reading a report from a user seeing the same) that it occurred to me to check the tokens like this. So my feeling is that it's probably been around a while.

I don't know if it's related, but just before I saw it, the Dart server responded slowly to a request for code actions which I had set for on-save, so I saw the popup like "Fetching code actions". It's possible I clicked cancel (since I was just trying to save the file and new no code actions would make changes). I recall also seeing some cancelled LSP requests in the log, though I assumed they were just from me moving around the files and could be unrelated.

Next time I see it, I'll try to take note of what happened (and review the logs) to see there might be any pattern. It certainly seems odd for the colours in the editor and this popup not to match, but I'm not familiar with how the implementation here works.

[DanTup]

@hediet I think I was able to reproduce this a few times by:

• typing quickly
• hitting Ctrl+Z to undo (probably before the last semantic tokens request has completed?)

I'm struggling to repro on demand though so those steps may have been coincidence.

I can't be certain that the Dart server isn't returning incorrect info (eg. something is getting out of order during async handling of changing the file and responding to semantic tokens), however I would that a) VS Code will cancel and re-issue a semantic token request after the last edit (so any inconsistency would be short-lived) and b) the popup shown in the screenshots above would be consistent with the colouring.

[DanTup]

I just noticed this again after changing Git branch. I changed git branch in GitHub desktop, which brought my working changes onto the new branch, then I switched back to VS Code and it looked like this. I can't be certain the issue occurred during the branch, but I think so.