Improve workflow for enabling/disabling semantic tokens

[findleyr]

Right now, to enable semantic tokens with gopls, it is required to set "gopls": { "semanticTokens": true }. Without this setting, gopls will return empty responses to semanticTokens requests (previously it returned an error, which led to the very verbose logs reported in golang/go#65712).

This is a bit backward: if gopls advertises the semanticTokens capability, and an LSP client requests semanticTokens, gopls should provide semanticTokens as best it can. If clients don't want semanticTokens, they should not request them.

We'd like to fix this by changing the default value of the hidden "semanticTokens" setting in gopls to "true". After this change, gopls will behave as expected by default: if clients ask for semantic tokens, they will get them.

However, if we make this change to gopls, it will implicitly enable semantic token for all VS Code users. This is a very large and observable change, and it is likely that many users will not want this behavior. For example, I personally do not want semantic tokens as I find them to be visually distracting.

We should update vscode-go so that users will still opt-in to semantic tokens, even after this gopls change. Should we change the default behavior of the VS Code LSP client to not request semantic tokens? Should we inject {"semanticTokens": false} into the vscode-go LSP settings, if it is unset?

CC @hyangah

[gopherbot]

Change https://go.dev/cl/579337 mentions this issue: gopls/internal/settings: enable semantic tokens by default

[hyangah]

@findleyr Can we mark this gopls setting deprecated or to-be-deprecated to reduce confusion?

[findleyr]

@hyangah yes, we can do that.
I'd like to also experiment with a minimal syntax-only implementation of semantic tokens, as currently I find them to be distractingly laggy (particularly following a metadata-affecting change). In that case, we may want to repurpose the "semanticTokens" setting, or create a new "semanticTokensStyle" setting.