Code completion items are duplicated and in alphabetical order

[j-michaels]

Describe the bug

I'm seeing two issues with code completion. Apologies if this should have been two separate tickets, but the solution I have for both is in the same part of code.

1. Completion items are getting sorted in alphabetical order. However, if you turn on verbose logging for sourcekit-lsp and look at what comes back from textDocument/completion, you can see that the ordering of completion items is not alphabetical, and the top results are likely to match with contextual information such as a function return type, and thus be more useful. The problem is that this extension doesn't have a handler defined for completion items, and it seems that VS Code's default behavior here is to sort on the label field of the CompletionItem. This field is populated with symbol's text itself, so that's just alphabetical order.

2. There are a lot of duplicate completion items. If you turn on verbose logging, you can see these duplicates in the sourcekit-lsp logs coming back for textDocument/completion. Each duplicate entry is an identical object in terms of content, but the order of keys is often different. The duplicates could be a bug in sourcekit-lsp, but I thought that reporting it here would be a good place to start, in case it was due to a bug in how this extension is using sourcekit-lsp.

To Reproduce

These two issues should be reproducible from any project by just triggering code completion. However, I'm able to reproduce it every time by doing this:

1. Ensure vapor is installed.
2. Run vapor new repro-bug -y and select postgres.
3. Open the project in VS Code, update packages and build it.
4. Trigger code completion, for example here's what it shows for the letter t in TodoController.swift on line 23:

But here's the completion list order from textDocument/completion in the logs:

true
todo
Todo
TodoDTO
TodoController
try
try!
try?
throw

Environment

• OS: macOS Sonoma 14.1
• Swift version: swift-driver version: 1.90.11.1 Apple Swift version 5.10 (swiftlang-5.10.0.13 clang-1500.3.9.4)
• Visual Studio Code version: 1.89.0
• vscode-swift version: 1.9.0

Possible Solution

I've worked out a solution for the sorting issue, and a band-aid for the duplicates, via a completion item provider in the middleware section of clientOptions.
https://github.com/swift-server/vscode-swift/blob/main/src/sourcekit-lsp/LanguageClientManager.ts#L438

provideCompletionItem: async (
	document: vscode.TextDocument,
	position: vscode.Position,
	context: vscode.CompletionContext,
	token: vscode.CancellationToken,
	next: langclient.ProvideCompletionItemsSignature
) => {
    const list = await next(document, position, context, token);
    if (!list) {
        return list;
    }
    const items = Array.isArray(list) ? list : list.items;
    // De-duplicate the completion item list using a hash
    const seen: Record<string, boolean> = {};
    const dedupItems = items.filter((item) => {
        const k = JSON.stringify(item);
        // eslint-disable-next-line no-prototype-builtins
        if (seen.hasOwnProperty(k)) {
            return false;
        } else {
            seen[k] = true;
            return true;
        };
    });
    // Change the sortText field to the index of each completion item, making sure
    // to pad it with leading 0s so the sort order is correct for numbers.
    for (let i = 0; i < dedupItems.length; i++) {
        dedupItems[i].sortText = `${i}`.padStart(5, '0');
    }
    if (!Array.isArray(list)) {
        list.items = dedupItems;
        return list;
    } else {
        return dedupItems;
    }
},

The above seems to have fixed the problems in my local, but this is the first time I've worked on VS Code extensions, for Swift or any language, so I'm pretty unfamiliar with all of this. Will it work as a solution?

[adam-fowler]

I've never seen duplicates before. Do you know where those symbols telBadRate etc are coming from

[adam-fowler]

Ok the sortText should probably be something that SourceKit-LSP sets.

[adam-fowler]

Here is an issue about the sorting apple/sourcekit-lsp#1234.

[j-michaels]

I've never seen duplicates before. Do you know where those symbols telBadRate etc are coming from

It looks like they're coming from header files in the macOS SDK, such as these:

/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/CarbonCore.framework/Versions/A/Headers/MacErrors.h
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/netinet/tcp.h
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/hfs/hfs_format.h

[adam-fowler]

OK I got slightly further with this now. The duplicate entries are due to Vapor exporting all the symbols from Foundation and SourceKit-LSP not expecting this. You will find this in the Vapor codebase

@_exported import Foundation

I've added another issue to the SourceKit-LSP apple/swift#74076