ycmd-docker

An example of running a containerised ycmd to:

• obviate the need to build ycm_core.so.
• capture completion suggestions for libraries that may not be present on the user's system when using YouCompleteMe.

Motivation

YouCompleteMe is a popular Vim plugin that provides the user with semantic completion. It has a client-server architecture, with the server component ycmd providing code-completion and comprehension. However, there is a pain point in that ycmd requires the user compile ycm_core.so with a modern enough CMake and LibClang for it to function properly.

Furthermore, more and more projects are using Docker containers to provide a uniform build environment containing all dependencies needed to build them. These dependencies (usually libraries and headers) may not be present on the user's system, and thus YouCompleteMe will not be able to pick them up for completion.

Implementation

Normally, YouCompleteMe fires up ycmd by invoking python on its nested ycmd folder (which essentially invokes the contained __main__.py). We will configure YouCompleteMe to instead fire up our own Docker container by fooling it into thinking our script ycmd-python is the Python server interpreter (when it really just launches our own dockerized ycmd instead).

Docker

The Dockerfile provided is only intended as an example (though it can be used for basic completion). The parent image ubuntu should instead be replaced with the build image, and subsequently apt-get with the corresponding package manager for that image.

Vim

We copy the ycmd-python script somewhere into our $PATH, tweaking the image name as necessary.

We then override g:ycm_server_python_interpreter to launch our own container, using an image which has both our dependencies (headers, libraries, etc) and ycmd:

Plug 'Valloric/YouCompleteMe'
let g:ycm_server_python_interpreter = 'ycmd-python'

It is imperative that the name of ycmd-python ends with python (should it need to be changed), else YouCompleteMe will refuse to launch it.

Emacs / Spacemacs

NB: I have little success with this; while in theory this should work emacs-ycmd fails to contact the launched Docker container.

Emacs integration with ycmd is provided through emacs-ycmd. Spacemacs provides this package through its ycmd layer.

To use our container here, we override ycmd-server-command to use our binary:

(setq ycmd-server-command '("ycmd-python")')

There does not appear to be any restriction on the name of ycmd-python, though filename expansion is not supported for characters like ~, so it may be necessary to use file-truename to expand it.

Diagnostics

Locating the container instance

A typical YouCompleteMe instance will display the following diagnostics with :YcmDebugInfo:

Printing YouCompleteMe debug information...
...
-- Server running at: http://127.0.0.1:8888
-- Server process ID: 12345
...

Normally we would use the process ID to locate the running instance and monitor it. However, as we are using docker to run this, we cannot use this method. To make this easier for the user, the containers launched using ycmd-python will have the format ycmd-<pid>, so that this will match what we see in the diagnostic information. In this instance, we would look for the container ycmd-12345 using docker ps -a.

Retrieving logs

A regular installation of YouCompleteMe will start a server that logs to several files, which can be inspected using :YcmDebugInfo:

Printing YouCompleteMe debug information...
...
-- Server running at: http://127.0.0.1:8888
...
-- Server logfiles:
--   /tmp/ycmd_8888_stdout_abcdefgh.log
--   /tmp/ycmd_8888_stderr_ijklmnop.log
...

However, on a long running server this will result in log files that may never be cleaned up.

To resolve this, we do not copy across the arguments provided to the stdout and stderr options so that ycmd will log to /dev/stdout and /dev/stderr respectively, and clean this up when we are done with them. To retrieve them, we can use docker logs on the particular instance that we want (using the process ID as described above).

Note that YouCompleteMe will still create these files regardless of whether they end up being used, though they will remain empty.

Caveats

Python completion

YouCompleteMe uses jedi for Python semantic completion. By default, it uses the same python interpreter used to run ycmd (which in the example image is /usr/bin/python3).

In order to capture completions for third-party libraries found in a virtual environment, one can tweak their .vimrc to point to the currently active python:

let g:ycm_python_binary_path = 'python'

ycmd-python will automatically pick this up if the activated virtual environment lies in $HOME.

:YcmCompleter GoTo

You will not be able to jump to a file that is located only inside the container, as your editor will not be able to find it in the host filesystem.

Updating

Each version of YouCompleteMe is designed for a particular ycmd revision. As such, every time we update our YouCompleteMe instance this image should also be updated accordingly.

This is done by tweaking the YCMD_REVISION variable in the Makefile to match the corresponding version used in YouCompleteMe.

Furthermore, we should transfer any differences in the original examples/example_client.py and the samples found in examples/samples across into our repository.