Get initialization options from document uri

[ghentschke]

For some LS the determination of LS initialization options depends on the first uri to be opened and not on the first project or path. Therefor the
getInitializationOptionsFromUri(URI) method has been added to the StreamConnectionProvider interface. It will be called when getInitializationOptions(URI) returns null.

fixes #683 #684

[mickaelistria]

For some LS the determination of LS initialization options depends on the first uri to be opened and not on the first project or path

I don't really understand this pattern. Can you please elaborate at why this would be necessary? Initialization options are usually more stateless server options that the client want to override. They're not really suitable for per-document/per-URI content.

[ghentschke]

I don't really understand this pattern. Can you please elaborate at why this would be necessary? Initialization options are usually more stateless server options that the client want to override. They're not really suitable for per-document/per-URI content.

The use case comes from a clangd specialty. clangd made some ls protocol extensions. One of them is the ability to provide initializationOptions.fallbackFlags in the jsonrpc initialize method. To determine the fallback flags for clangd, the C/C++ compiler has to run on the first translation unit to be opened on a project/workspace. The compiler is asked for its builtin system includes.

Thats the reason why we need the initialUri during start() of the LS.

[mickaelistria]

Thanks for sharing the use-case. The use-case is valid so I think it's fine to make LSP4E capable of supporting it.
But overall, I'm still not confident it's the best path forward as it relies strongly on the idea that a LS is project-specific, which is not the best pattern currently. Can't those failback flags be detected server-side by the LS? Couldn't they be sent as configuration (https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#workspace_didChangeConfiguration )? Is there a plan for clangd to support multiple workspace folders (so another solution will be necessary)?

[ghentschke]

I'm still not confident it's the best path forward as it relies strongly on the idea that a LS is project-specific

Regarding this issue it's not important whether the LS is project specific or not. It no change of configuration. Its all about initial initialization settings. All we want to know is the first file/document which triggers the LS to run. We'll determine the compiler settings for this file and tell clangd where to find the system include headers for the case there is no compile_commands.json provided/found.

This is independent of the amount of LS.

[mickaelistria]

All we want to know is the first file/document which triggers the LS to run

Isn't it the first doc for which didOpen is sent? So this info is redundant.

We'll determine the compiler settings for this file

Can't clangd detect that? (it's a naive question, but I need to ask it as we should avoid seeing the language clients host workarounds for features that would be much more valuable placed in the server)

where to find the system include headers for the case there is no compile_commands.json provided/found.
This is independent of the amount of LS.

Do all files/projects use the same system headers? If yes, then you most likely don't need to pass the initial doc to resolve such a static value; if not then it means for distinct files, you should use distinct values; so either you have multiple LS receiving each a distinct value, or you have a single LS which dynamically receives the possible valueS (according to the open file). I don't think initialization option fits any of this case, IMO clangd could do better here supporting configuraiton, particularly since it seems to be already capable of dealing with workspaceFolders.

[ghentschke]

Isn't it the first doc for which didOpen is sent? So this info is redundant.

didOpen is too late and there could be several async didOpen commands. This was my first approach. The only reliable way is to do the settings in the initialize phase of the LS:

For clangd it looks like this (command with included initializationOptions:

[2023-06-19T10:08:27.631966+02:00] LSP4E_TO_LANGUAGE_SERVER org.eclipse.cdt.lsp.server:
{"jsonrpc":"2.0","id":"1","method":"initialize","params":{"processId":40448,"rootPath":"/C:/msys2/msys64/mingw64/include/c++/12.2.0/","rootUri":"file:///C:/msys2/msys64/mingw64/include/c++/12.2.0/","initializationOptions":{"fallbackFlags":["-isystemC:/TR/workspaces/runtime-cdt-development/Cmk3/build/cmake.debug.win32.x86_64","-isystemC:/msys2/msys64/mingw64/include/c++/12.2.0","-isystemC:/msys2/msys64/mingw64/include/c++/12.2.0/x86_64-w64-mingw32","-isystemC:/msys2/msys64/mingw64/include/c++/12.2.0/backward","-isystemC:/msys2/msys64/mingw64/lib/clang/15.0.7/include","-isystemC:/msys2/msys64/mingw64/include"]},"capabilities":{"workspace":{"applyEdit":true,"workspaceEdit":{"documentChanges":true,"resourceOperations":["create","delete","rename"],"failureHandling":"undo"},"symbol":{"dynamicRegistration":true},"executeCommand":{"dynamicRegistration":true},"workspaceFolders":true,"configuration":true,"codeLens":{"refreshSupport":true}},"textDocument":{"synchronization":{"willSave":true,"willSaveWaitUntil":true,"didSave":true},"completion":{"completionItem":{"snippetSupport":true,"documentationFormat":["markdown","plaintext"],"resolveSupport":{"properties":["documentation","detail","additionalTextEdits"]},"insertTextModeSupport":{"valueSet":[1,2]}}},"hover":{"contentFormat":["markdown","plaintext"]},"signatureHelp":{},"references":{},"documentHighlight":{},"documentSymbol":{"symbolKind":{"valueSet":[18,17,5,14,9,10,22,24,8,1,12,11,20,6,2,3,21,16,19,25,4,7,15,23,26,13]},"hierarchicalDocumentSymbolSupport":true},"formatting":{"dynamicRegistration":true},"rangeFormatting":{},"definition":{"linkSupport":true},"typeDefinition":{"linkSupport":true},"codeAction":{"codeActionLiteralSupport":{"codeActionKind":{"valueSet":["quickfix","refactor","refactor.extract","refactor.inline","refactor.rewrite","source","source.organizeImports"]}},"dataSupport":true,"resolveSupport":{"properties":["edit"]},"dynamicRegistration":true},"codeLens":{},"documentLink":{},"colorProvider":{},"rename":{"prepareSupport":true},"foldingRange":{},"selectionRange":{},"inlayHint":{}},"window":{"workDoneProgress":true,"showMessage":{},"showDocument":{"support":true}}},"clientInfo":{"name":"Eclipse Platform","version":"0.17.0.qualifier"},"trace":"off","workspaceFolders":[{"uri":"file:///C:/TR/workspaces/runtime-cdt-development/Cmake5/","name":"Cmake5"},{"uri":"file:///C:/TR/workspaces/runtime-cdt-development/Cmk1/","name":"Cmk1"},{"uri":"file:///C:/TR/workspaces/runtime-cdt-development/Cmk3/","name":"Cmk3"},{"uri":"file:///C:/TR/workspaces/runtime-cdt-development/CmkOldGcc/","name":"CmkOldGcc"},{"uri":"file:///C:/TR/workspaces/runtime-cdt-development/External%20Plug-in%20Libraries/","name":"External Plug-in Libraries"},{"uri":"file:///C:/TR/workspaces/runtime-cdt-development/ManagedBuild/","name":"ManagedBuild"},{"uri":"file:///C:/TR/workspaces/runtime-cdt-development/ManagedCpp/","name":"ManagedCpp"},{"uri":"file:///C:/TR/workspaces/runtime-cdt-development/MyCmake/","name":"MyCmake"},{"uri":"file:///C:/TR/workspaces/runtime-cdt-development/NewCmake/","name":"NewCmake"},{"uri":"file:///C:/TR/workspaces/runtime-cdt-development/hello/","name":"hello"}]}}

Can't clangd detect that?

Unfortunately not, because they are fallback settings for clangd. Under normal circumstances clangd works like this: It searches in the folder of the C/C++ file ( (which should be opened in the editor) for a compile_commands.json file with compiler info for the file to be opened. It this commands file can't be found, the parent folders will be searched by clangd. This works fine for source files in the project or workspace. It fails for system header files e.g. iostream which are located near the compiler. On windows the compiler can be anywhere on your machine (see attached screenshot for an example):

After an IDE restart, with a system header file in focus (as shown in the screenshot above), clangd simply has no clue where to find the other included system header files in a system header itself: The link in iostream to bits/c++config.h cannot be resolved by clangd without knowledge of the system include path.

In this case clangd relies on its fallback settings. Therefor we have to set these fallbacks in the initializationOptions in the initialize sequence. We determine these path by running the compiler with special options on the first file to be opened and parsing the output for system headers path. That's the reason why clangd made these settings available in the initialize phase.

Do all files/projects use the same system headers?

That's not the problem here. As stated above, we need to determine fallbackFlags fot the case when no compile_commands.json is available for a file to be opened (that can be the case for already opened system headers after an IDE restart.)

you have a single LS which dynamically receives the possible valueS (according to the open file)

That's the regular case and works fine according to the mechanism I described above.

[angelozerr]

@ghentschke even i fyou have not the intention to integrate clangd in another LSP client, you should think about that to follow LSP protocol. IMHO your PR is very specific for clangd. You push the the URI of the first opened file, but perhaps an another language server would like to know what are all opened files (ex : you open several files and you close your workspace and you reopen your workspace and the LS need to know what are all opened files).

If I understand correctly your requirement, you need to push the first opened file in initialized parameters. In this case, when I will build the initialize parameters, I will use the Eclipse IDE API to know the activated editor and get the opened file. But perhaps it is not possible, I have not tested.

[mickaelistria]

@ghentschke my concerns are the ones expressed by @angelozerr that this changes seem very specific to the current state of clangd. The discussion is intended to find whether we can make clangd integration work without this change, and discuss other appraches for the integration; or whether the problem you're facing for clangd is a more generic one and discuss whether the solution is the most generic.

A language-server can be initialized out of a document context, just started to be ready with doc bound to it (yet); expecting a document here seems too early. LSP has the concept of rootUri that is used on initialization. Couldn't this be used instead of document URI to create initialization options?
Also, do all project use the same failback system headers? Or does it really depend on the current file/project/rootUri...?

[ghentschke]

@angelozerr and @mickaelistria : Thank you for the feedback! I'am fully aware that this proposal is very specific to clangd. I can understand the concerns.
What I need is: the first URI to be opened (I stated the reason in my post above). Unfortunately this info is hold by LSP4E only, because during the IDE startup, the LSP4E extensions for the generic text editor gets involved and triggers the LS to be started.

At the moment I've no better idea how to fetch the initialFile from LSP4E.

I will use the Eclipse IDE API to know the activated editor and get the opened file

When I remember it correctly, this will not work if the focus is not on the editor part, but I'll give it another try.

[angelozerr]

In vscode it seems that you store fallbackFlags in a settings https://github.com/clangd/vscode-clangd/blob/d0653e60b0cce91b87a55e6321f9cb442cdf5a9d/src/clangd-context.ts#L86 so even if you fix your problem with LSP4E you will have the same problem with vscode.

Perhaps you could add a tracker of editor and stores the fallbackFlags in a memory or in a preferences the first file which is opened and after that you could use this information in InitialiazedParams?

[ghentschke]

so even if you fix your problem with LSP4E you will have the same problem with vscode

The goal here is to be better than vscode ;-) CDT shall handle this flags dynamically.

Perhaps you could add a tracker of editor and stores the fallbackFlags in a memory or in a preferences the first file which is opened and after that you could use this information in InitialiazedParams?

I do something similar in my current (hacky) solution: I check whether the first URI, which I get from the LSP4E enableWhen test is part of a workspace project. If it is, I use this URI as my initial URI and store it persistent. During the next startup I use this URI to determine the callback flags.
The drawback of this solution is, that this check in a property tester is a little hacky.

[mickaelistria]

@ghentschke I more and more believe that this failbackFlags in general is not suitable in the initialize step, for the reason mentioned earlier that initialize can totally happen without a document being open. The coupling you're building between document being open and intialization is IMO an anti-pattern. The rootUri is the only path that makes sense during intialization, we could use it as a parameter, but the 1st document open is not something we can reliably rely on. Would passing the rootUri to getIntiializationOptions work?

I understand that it's the current state of clangd, but then I wonder, wouldn't it be more profitable, for all integrations of clangd, to contribute there the ability to define such a path with standard LSP approaches such as configuration (which you could then send programatically in a workbench listener when an editor is activated), or with a custom protocol extension...?
I thnk both approach would serve your goal better, without need to bend LSP4E and other clients; and has the benefits of scaling better as well when clangd evolves and makes that it supports multiple workspaceFolders, requires different default system headers location (1 for each project) and so on.

[mickaelistria]

The initial target branch master was deleted and replaced by main, so this PR got closed automatically. If this is still relevant, please re-create this PR targetting the main branch.