Completion breaks with multibyte characters #427

arp242 · 2021-06-20T03:02:02Z

The following file:

package x

import (
    "fmt"
    "net/mail"
)

func t() {
    var m mail.Address
    fmt.Printf("%s → %s\n", m.)
}

Completing the m doesn't work in lsc. I tried VSCode, and it works there, so it seems a problem in vim-lsc rather than gopls.

What it sends to gopls is:

{
  "id": 2,
  "jsonrpc": "2.0",
  "method": "textDocument/completion",
  "params": {
    "position": {
      "character": 29,
      "line": 9
    },
    "textDocument": {
      "uri": "file:///tmp/tgo_20210620_w6ByRcao/main.go"
    }
  }
}

Putting the cursor right on top of the m and starting completion does work. Replacing that → (U+2192) with e.g. > also fixes it. So it seems it should send "character": 27 instead of "character": 29.

Looking at the LSC specification, character offsets are kinda weird:

The offsets are based on a UTF-16 string representation. So a string of the form a𐐀b the character offset of the character a is 0, the character offset of 𐐀 is 1 and the character offset of b is 3 since 𐐀 is represented using two code units in UTF-16. To ensure that both client and server split the string into the same line representation the protocol specifies the following end-of-line sequences: ‘\n’, ‘\r\n’ and ‘\r’.

Microsoft's love for UTF-16 strikes again 🙃

Right now it looks like it just sends the byte offset(?)

The text was updated successfully, but these errors were encountered:

arp242 · 2021-06-20T03:17:49Z

Sorry, I forgot to mention I'm using the latest master (4b0fc48)

I don't know what you comparability requirements are for vim-lsc, but Vim somewhat recently added some multibyte-aware functions to make all of this easier, e.g.:

:echo charidx(getline('.'), 29)
27

This requires Vim 8.2.2233 though (Dec 2020), and it'll be quite some time before e.g. Debian people will get this, but you can use e.g. has() to check for it. Older Vims can get the broken behaviour, new Vims the new behaviour.

Also see charcol().

I couldn't really figure out the logic in completion.vim right now or I'd send a patch 😅 But with some luck it should be a comparatively simple fix(?)

Closes #427 Use `charcol` over `col` where it exists. Gate the function definition behind the feature check to avoid repeating it for every call.

natebosch · 2021-06-24T01:25:03Z

@arp242 - Thanks for the pointer to charcol!

Could you try the multibyte-position branch and confirm that it solves your issue? From my debugging it seems like it should, but I think the server I'm running against might have a complimentary bug.

arp242 · 2021-06-24T02:38:55Z

Thanks! It seems improved, but it doesn't display all completions:

I checked the LSP logs, and the request and responses are completely identical for the two examples (x and →), so it seems this part is correct, but somehow something else is still causing problems? 🤔

I tested with a clean vimrc as well.

Logs for completeness sake (but I don't think this is where the problem is):

{
  "id": 2,
  "jsonrpc": "2.0",
  "method": "textDocument/completion",
  "params": {
    "position": {"character": 26, "line": 11},
    "textDocument": {"uri": "file:///tmp/tgo_20210624_hs3UF4yn/main.go"}
  }
}

{
  "id": 2,
  "jsonrpc": "2.0",
  "result": {
    "isIncomplete": true,
    "items": [
      {
        "detail": "string",
        "documentation": "user@domain\n",
        "filterText": "Address",
        "insertTextFormat": 1,
        "kind": 5,
        "label": "Address",
        "labelDetails": {},
        "preselect": true,
        "sortText": "00000",
        "textEdit": {
          "newText": "Address",
          "range": {"end": {"character": 26, "line": 11}, "start": {"character": 26, "line": 11}}
        }
      },
      {
        "detail": "string",
        "documentation": "Proper name; may be empty.\n",
        "filterText": "Name",
        "insertTextFormat": 1,
        "kind": 5,
        "label": "Name",
        "labelDetails": {},
        "sortText": "00001",
        "textEdit": {
          "newText": "Name",
          "range": {"end": {"character": 26, "line": 11}, "start": {"character": 26, "line": 11}}
        }
      },
      {
        "detail": "func() string",
        "documentation": "String formats [..].\n",
        "filterText": "String",
        "insertTextFormat": 1,
        "kind": 2,
        "label": "String",
        "labelDetails": {},
        "sortText": "00002",
        "textEdit": {
          "newText": "String",
          "range": {"end": {"character": 26, "line": 11}, "start": {"character": 26, "line": 11}}
        }
      }
    ]
  }
}

commit 8851e05 Author: Nate Bosch <nbosch1@gmail.com> Date: Wed Jun 23 18:20:09 2021 -0700 Fix cursor position following multibyte characters Closes natebosch#427 Use `charcol` over `col` where it exists. Gate the function definition behind the feature check to avoid repeating it for every call.

natebosch added a commit that referenced this issue Jun 24, 2021

Fix cursor position following multibyte characters

8851e05

Closes #427 Use `charcol` over `col` where it exists. Gate the function definition behind the feature check to avoid repeating it for every call.

natebosch linked a pull request Jun 24, 2021 that will close this issue

Fix cursor position following multibyte characters #428

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Completion breaks with multibyte characters #427

Completion breaks with multibyte characters #427

arp242 commented Jun 20, 2021 •

edited

arp242 commented Jun 20, 2021

natebosch commented Jun 24, 2021

arp242 commented Jun 24, 2021 •

edited

Completion breaks with multibyte characters #427

Completion breaks with multibyte characters #427

Comments

arp242 commented Jun 20, 2021 • edited

arp242 commented Jun 20, 2021

natebosch commented Jun 24, 2021

arp242 commented Jun 24, 2021 • edited

arp242 commented Jun 20, 2021 •

edited

arp242 commented Jun 24, 2021 •

edited