feat: Add native lima Docker provider support #5892

rfay · 2024-02-23T16:54:49Z

The Issue

Colima has gotten quite unstable for lots of people, but it's just a wrapper on Lima, and so we should be able to use Lima directly as another open-source alternative.

How This PR Solves The Issue

It's worth considering whether we should do this at all, as adding more options doesn't necessarily help people or our support environment.

TODO

Docs
Turn on tests (preferably without putting red marks on other PRs)
Make it pass tests

Manual Testing Instructions

Use the docs to start a lima instance, then do a simple quickstart test.

Automated Testing Overview

Release/Deployment Notes

github-actions · 2024-02-25T22:24:24Z

Download the artifacts for this pull request:

See Testing a PR

rfay · 2024-03-08T19:44:03Z

Fails TestDdevAllDatabases on tb-macos-arm64-7 with non-writable filesystem:
https://buildkite.com/ddev/macos-lima/builds/12#018e1f6d-091a-4022-8753-f14e26bb1c74

TestWritableFilesDirectory on tb-macos-arm64-6: https://buildkite.com/ddev/macos-lima/builds/18#018e2142-c31e-405c-9db3-3a416e2790f1

rpkoller

just left a few comments early on. just after leaving comments on the colima quickstart i noticed that there are certain overlaps for those two and it might be a good thing to keep things consistent across quickstarts. and would it make sense since colima and rancher are based on lima to place lima in front of the two?

rpkoller · 2024-03-08T22:47:52Z

docs/content/users/install/docker-installation.md

@@ -46,6 +45,27 @@ You’ll need a Docker provider on your system before you can [install DDEV](dde
    !!!warning "Colima can only work in your home directory unless you do further configuration"
        By default, Colima only mounts your home directory, so it’s easiest to use it in a subdirectory there. See the `~/.colima/default/colima.yaml` for more information, or notes in [colima.yaml](https://github.com/abiosoft/colima/blob/fc948f8f055600986f87e29e3e632daf56ac8774/embedded/defaults/colima.yaml#L130-L143).

+    ### Lima
+
+    [Lima](https://github.com/lima-vm/lima) is a free and open-source project supported by the [Cloud Native Computing Foundation](https://cncf.io/).


one note. colima currently states that bundles a container management tool called [Lima](https://github.com/lima-vm/lima) with a Docker (Linux) backend.. with lima not an option in this list it was ok to phrase it that way but as soon as lima is added i would rephrase it. and i would also add a note to rachner that it is based on lima. that has no note about that yet.

Sorry, can't understand your question but feel free to propose something.

rpkoller · 2024-03-08T22:56:28Z

docs/content/users/install/docker-installation.md

+
+    [Lima](https://github.com/lima-vm/lima) is a free and open-source project supported by the [Cloud Native Computing Foundation](https://cncf.io/).
+
+    1. Install Lima with `brew install lima`.


I would make step 1 and 2 consistent with Colima.

Run docker help to verify if the Docker client is installed. In case of an error, install it Homebrew by running brew install docker (slightly altered the Colima version might be adjusted accordingly?)

Install Lima with brew install lima

I like it better this way, didn't like the other one as much.

docs/content/users/install/docker-installation.md

rpkoller · 2024-03-08T23:16:06Z

docs/content/users/install/docker-installation.md

+    2. Install the `docker` client if you don't already have it with `brew install docker`.
+    3. Create a default Lima setup with 4 CPUs, 6GB memory, 100GB storage, and Cloudflare DNS, adjusting as needed:
+    ```
+    limactl create --vm-type=vz --mount-type=virtiofs --mount-writable --memory=6 --cpus=4 --disk=100 template://docker


i am cautious and a bit afraid of mounting the entire home directory, most of the lima maintainers share the same worry. That is the reason why out of the box only the lima temp directory is added. but the rancher maintainer is the only lima maintainer who mounts his entire homedirectory for convenience reasons. just as a note. i am ok with it but personally i only use --mount="~/Sites:w" (with Sites being the directory i store all my projects)

I understand that your caution, but this is the default behavior and doing it otherwise creates enormous support problems.

rpkoller · 2024-03-08T23:18:08Z

docs/content/users/install/docker-installation.md

+    3. Create a default Lima setup with 4 CPUs, 6GB memory, 100GB storage, and Cloudflare DNS, adjusting as needed:
+    ```
+    limactl create --vm-type=vz --mount-type=virtiofs --mount-writable --memory=6 --cpus=4 --disk=100 template://docker
+    docker context create lima-default --docker "host=unix://$HOME/.lima/default/sock/docker.sock"


it probably should be noted that this command has only to be run once after the initial creation of the VM. and this line as well as the next (docker context use) are also output by default. they are part of the messagethat is defined in the instances yaml file.

This is shown as part of the first-time setup.

rpkoller · 2024-03-08T23:19:08Z

docs/content/users/install/docker-installation.md

+    docker context create lima-default --docker "host=unix://$HOME/.lima/default/sock/docker.sock"
+    docker context use lima-default
+    ```
+    After the initial run above, you can use `limactl start`.  Run `limactl list` to see configured setup.


Suggested change

After the initial run above, you can use `limactl start`. Run `limactl list` to see configured setup.

After the initial setup, you can start your instance with `limactl start`. Run `limactl --help` to see the list of available commands.

instead of adding the command to see the configured setup i think providing the help command to get the list of all available command might be more helpful (which also includes the list command)

I like it as is. People have to branch out to learn more.

rpkoller · 2024-03-08T23:26:08Z

docs/content/users/install/docker-installation.md

+    !!!warning "Docker contexts let the Docker client point at the right Docker server"
+        The Docker provider you're using is selected with `docker context`. You can see the available contexts with `docker context ls` and the currently selected one with `docker context show`. With the setup above you'll want `docker context use lima-default`.
+
+    !!!warning "Lima only mounts filesystems in your home directory unless you do further configuration"


same as for the colima quickstart. i think it would make sense to clarify that those mounts are not necessary for lima to run but for ddev projects to run?

Here we're only interested in people running DDEV projects. People can do other things with Lima if they want to.

If people don't put their projects on a filesystem that Lima mounts then DDEV is nonfunctional. Most people put their projects somewhere in the homedir.

rfay

Thanks for taking such a good look @rpkoller and thanks for all your expertise in this area!

rfay · 2024-03-09T00:26:33Z

docs/content/users/install/docker-installation.md

+    2. Install the `docker` client if you don't already have it with `brew install docker`.
+    3. Create a default Lima setup with 4 CPUs, 6GB memory, 100GB storage, and Cloudflare DNS, adjusting as needed:
+    ```
+    limactl create --vm-type=vz --mount-type=virtiofs --mount-writable --memory=6 --cpus=4 --disk=100 template://docker


I understand that your caution, but this is the default behavior and doing it otherwise creates enormous support problems.

rfay · 2024-03-09T00:26:52Z

docs/content/users/install/docker-installation.md

+    3. Create a default Lima setup with 4 CPUs, 6GB memory, 100GB storage, and Cloudflare DNS, adjusting as needed:
+    ```
+    limactl create --vm-type=vz --mount-type=virtiofs --mount-writable --memory=6 --cpus=4 --disk=100 template://docker
+    docker context create lima-default --docker "host=unix://$HOME/.lima/default/sock/docker.sock"


This is shown as part of the first-time setup.

rfay · 2024-03-09T00:27:48Z

docs/content/users/install/docker-installation.md

+    docker context create lima-default --docker "host=unix://$HOME/.lima/default/sock/docker.sock"
+    docker context use lima-default
+    ```
+    After the initial run above, you can use `limactl start`.  Run `limactl list` to see configured setup.


I like it as is. People have to branch out to learn more.

rfay · 2024-03-09T00:29:23Z

docs/content/users/install/docker-installation.md

+    !!!warning "Docker contexts let the Docker client point at the right Docker server"
+        The Docker provider you're using is selected with `docker context`. You can see the available contexts with `docker context ls` and the currently selected one with `docker context show`. With the setup above you'll want `docker context use lima-default`.
+
+    !!!warning "Lima only mounts filesystems in your home directory unless you do further configuration"


Here we're only interested in people running DDEV projects. People can do other things with Lima if they want to.

If people don't put their projects on a filesystem that Lima mounts then DDEV is nonfunctional. Most people put their projects somewhere in the homedir.

Co-authored-by: Ralf Koller <1665422+rpkoller@users.noreply.github.com>

github-actions bot added the enhancement label Feb 23, 2024

rfay force-pushed the 20240222_lima branch from 03a8eb4 to 8d0b1dd Compare February 25, 2024 20:37

rfay force-pushed the 20240222_lima branch from 8d0b1dd to 3baf683 Compare February 28, 2024 18:43

rfay added 3 commits March 8, 2024 10:14

feat: basic support for lima docker provider

1f5f4e7

don't run tests right now [skip ci]

732c424

Add lima docs

da35f86

rfay force-pushed the 20240222_lima branch from 3baf683 to da35f86 Compare March 8, 2024 18:55

turn on lima testing

564a61c

rpkoller reviewed Mar 8, 2024

View reviewed changes

rfay added 2 commits March 8, 2024 16:46

TestDdevAllDatabases should test key mariadb versions most of the time

fde3ed5

Make test.sh more robust running manually [skip ci]

0657061

rfay commented Mar 9, 2024

View reviewed changes

rfay and others added 5 commits March 8, 2024 17:32

@rpkoller suggestion [skip ci]

de8860b

Co-authored-by: Ralf Koller <1665422+rpkoller@users.noreply.github.com>

try to capture broken project

e13b3bf

Minor docs in app_compose_template

ec13cbc

Don't run TestDebugNFSMount on Lima

7e5f0ad

Skip a number of tests on lima

9040cf4

rfay mentioned this pull request Mar 9, 2024

TestAutocompletionForCustomCmds often fails on various platforms: /mnt/ddev_config/commands/web/autocomplete/project-web-cmd: Permission denied #5904

Closed

Skip more tests on lima/colima

60d76f9

rfay mentioned this pull request Mar 9, 2024

TestDdevAllDatabases sometimes failing (colima) with the wrong timezone #5912

Closed

stop checking nfs on ports check

7bec8b3

rfay marked this pull request as ready for review March 11, 2024 13:35

rfay requested review from a team as code owners March 11, 2024 13:35

incorporate @rpkoller feedback [skip ci]

fd00440

rfay merged commit 60831ee into ddev:master Mar 11, 2024
2 of 8 checks passed

rfay deleted the 20240222_lima branch March 11, 2024 13:52

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Add native lima Docker provider support #5892

feat: Add native lima Docker provider support #5892

rfay commented Feb 23, 2024 •

edited

github-actions bot commented Feb 25, 2024 •

edited

rfay commented Mar 8, 2024 •

edited

rpkoller left a comment

rpkoller Mar 8, 2024

rfay Mar 9, 2024

rpkoller Mar 8, 2024

rfay Mar 9, 2024

rpkoller Mar 8, 2024

rfay Mar 9, 2024

rpkoller Mar 8, 2024

rfay Mar 9, 2024

rpkoller Mar 8, 2024

rfay Mar 9, 2024

rpkoller Mar 8, 2024

rfay Mar 9, 2024

rfay left a comment

rfay Mar 9, 2024

rfay Mar 9, 2024

rfay Mar 9, 2024

rfay Mar 9, 2024


		[Lima](https://github.com/lima-vm/lima) is a free and open-source project supported by the [Cloud Native Computing Foundation](https://cncf.io/).

		1. Install Lima with `brew install lima`.

	After the initial run above, you can use `limactl start`. Run `limactl list` to see configured setup.
	After the initial setup, you can start your instance with `limactl start`. Run `limactl --help` to see the list of available commands.

feat: Add native lima Docker provider support #5892

feat: Add native lima Docker provider support #5892

Conversation

rfay commented Feb 23, 2024 • edited

The Issue

How This PR Solves The Issue

TODO

Manual Testing Instructions

Automated Testing Overview

Related Issue Link(s)

Release/Deployment Notes

github-actions bot commented Feb 25, 2024 • edited

rfay commented Mar 8, 2024 • edited

rpkoller left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rfay left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rfay commented Feb 23, 2024 •

edited

github-actions bot commented Feb 25, 2024 •

edited

rfay commented Mar 8, 2024 •

edited