feat: generalize HOMEDIR management and respect XDG_CONFIG_HOME, fixes #5807

[danepowell]

The Issue

• Support XDG base directory specification #5807

This is my first time touching Go so please "go" easy on me! 😄 Thanks for having great documentation, good code, and an easy build process that made it a pretty easy learning process.

How This PR Solves The Issue

1. Allows to move the global ~/.ddev directory to $XDG_CONFIG_HOME/ddev or predefined OS specific location:

   • ~/.config/ddev on Linux and WSL2 only

2. If there are multiple DDEV global configurations, they are checked with the following priority:

   1. $XDG_CONFIG_HOME/ddev
   2. ~/.config/ddev on Linux and WSL2 only
   3. ~/.ddev

   The current location of the configuration directory can be checked using ddev version | grep global-ddev-dir

3. Switches from ~/.ddev/mutagen_data_directory to ~/.ddev/.mdd

4. Adds check for Mutagen socket length. See:

   • Mutagen daemon fails to start if MUTAGEN_DATA_DIRECTORY is too long mutagen-io/mutagen#453
   • 0.13: [Bug]: in-cluster build sync error when root path is very long garden-io/garden#4527 (comment)
   • https://unix.stackexchange.com/a/367012

5. TODO (probably not in this PR):

   • Move the global config folder in Gitpod (if applicable) to a permanent location using XDG_CONFIG_HOME=/workspace/.config and mkdir -p $XDG_CONFIG_HOME/ddev

Manual Testing Instructions

1. Check moving ~/.ddev without using $XDG_CONFIG_HOME (on Linux and WSL2):

   1. Move ~/.ddev to OS specific config folder:
      • ~/.config/ddev on Linux and WSL2
   2. Check that old Mutagen folder ~/.ddev_mutagen_data_directory does exist.
   3. Run ddev list
   4. Run ddev version | grep global-ddev-dir to confirm that directory is different from ~/.ddev
   5. Check that ~/.ddev is not recreated
   6. Run ddev start
   7. Old ~/.ddev_mutagen_data_directory should be deleted.
   8. Move .ddev back to ~/.ddev and confirm that everything works as before

2. Repeat all steps from 1, but this time use export $XDG_CONFIG_HOME=/custom-path && mkdir -p $XDG_CONFIG_HOME/ddev

3. Check Mutagen error for really long path to its socket:

   1. ddev config global --performance-mode=mutagen
   2. check ddev mutagen version output
   3. Set $XDG_CONFIG_HOME to some really long path, in result the full path to $XDG_CONFIG_HOME/ddev/.mdd/daemon/daemon.sock should be at least 104 characters on macOS or 108 characters on Linux
   4. mkdir -p $XDG_CONFIG_HOME/ddev
   5. check ddev mutagen version output (the locations should be different).
   6. run ddev start and see a warning for too long path

Automated Testing Overview

Tests that changed $HOME are rewritten to use $XDG_CONFIG_HOME (except for TestCreateGlobalDdevDir, that needs to create a directory).

Related Issue Link(s)

Fixes #5807

Release/Deployment Notes

This PR doesn't introduce any breaking changes, it adds more options for moving the DDEV global config directory to another location.

[rfay]

Looking forward to test results. Will require excellent manual testing.

[rfay]

Static validation failures are due to upgrade of golanci-lint, will be fixed shortly in

• test: golangci-lint got pickier, satisfy it #5812

That should go in at least by tomorrow.

[danepowell]

Thanks. I'll circle back to this Monday to clean up tests and any other issues.

[github-actions]

Download the artifacts for this pull request:

• all-ddev-executables.zip
• ddev-linux-amd64.zip
• ddev-linux-arm64.zip
• ddev-macos-amd64.zip
• ddev-macos-arm64.zip
• ddev-windows-amd64-installer.zip

See Testing a PR.

[rfay]

@danepowell could you please PM me any way you want about some Acquia things? Discord probably best, but email and other contacts are on https://github.com/rfay

[rfay]

Looks like TestCreateGlobalDdevDir will need some attention.

When you rebase this the static validation will be fixed.

[danepowell]

The tests now pass locally (woot! 😄 )

The problem is they inherit certain properties of the host environment, namely the OS config dir. So I have to hardcode ~/Library/Application Support which will obviously break on a Linux runner. Not sure how to fix that yet.

[danepowell]

This should be ready for review. The tests are still a little flaky on my local but I think that's due to some kind of interference between the test fixtures and my host environment. It also seems like some of the failing tests in CI are unrelated to this PR.

[rfay]

One thing to know is that since it's your first PR, each push has to be approved for testing. So ping me if you push and need to see the new results. I see your pushes, so will try to make sure I start when it happens. Use [skip ci] in commit message if you don't want to run tests on a commit.

[GuySartorelli]

This works great! Uses the ~/.ddev/ dir if that exists (which it will for existing installations), but if that dir doesn't exist it'll create (or use if it exists) the ddev/ dir in the configured config dir.

Can't test for mac or windows, but works great in gitpod and locally on ubuntu.

The failing colima test is probably a red herring, that test likes to fail (@rfay I thought you'd given me access to rerun these but it looks like I can't? I can only rerun buildkite CI it looks like)

[GuySartorelli]

Just had a thought - there's a bunch of code comments (and maybe docs?) that refer to ~/.ddev - these should be updated to avoid confusion.

[rfay]

but works great in gitpod and locally on ubuntu.

One of the problems with Gitpod is that the homedir is not persistent. I think we could actually use this to solve that problem on gitpod! /cc @shaal . /workspace is what persists there, so we could use actually /workspace itself as $XDG_CONFIG_HOME. Huge win!

[rfay]

there's a bunch of code comments (and maybe docs?) that refer to ~/.ddev - these should be updated to avoid confusion.

I think there's a lot of little things that do need to be taken care of, https://github.com/search?q=repo%3Addev%2Fddev%20~%2F.ddev&type=code - most of them are not important but there are a few that are.

I'm OK with doing this as a followup, but some of those look really important.

The code situations (not comments) absolutely have to be taken care of.

[danepowell]

I hadn't considered how many (156, at least 😄 ) hardcoded references to ~/.ddev we'd need to deal with.

I don't think we actually have to fix any of those immediately. In fact, we might be better off waiting until the next major release, when/if support for ~/.ddev is actually removed, in order to fix most of those. Otherwise there's a chance we do all of that work and never end up actually deprecating ~/.ddev.

I'd suggest that the one exception is public-facing docs. We should update those in order to avoid confusion where new users go looking for ~/.ddev when it doesn't exist.

Let me know what you think.

[GuySartorelli]

I don't think we actually have to fix any of those immediately.

I'd strongly disagree with that. Who knows what random bugs will suddenly appear because someone does a fresh install, their config is put in the new place, but the hardcoded references are looking in the wrong place...

[rfay]

The ones in Dockerfiles aren't relevant.
list.go looks important
Location of router_compose.yaml is important
Location of mutagen_data_directory is critical
Location of ~/.ddev/bin is critical

I haven't looked through all of them, but it seems to me like we have more to do than originally intended.

[danepowell]

The problem is that some of those file are shell scripts and other non-Go libraries, so they can't take advantage of the same config directory logic. They'll need to use XDG_CONFIG_HOME and maybe even implement a fallback mechanism if it's not set.

I'll be honest, that takes this far beyond what I was hoping to accomplish as a "20% project". I'm also going to be out of office until March. I'll try to check back in when I return.

[rfay]

I think this is a great initiative. The ones that affect scripts are not all relevant, and the ones that take place inside containers are not relevant. But the golang ones that are for things other than ~/.ddev (like ~/.mutagen_data_directory) are obviously critical. Of course... moving .mutagen_data_directory into ~/.ddev would make a lot more sense.

Thanks so much for this work, and I hope it works out.

[rfay]

Moved back to draft so it can't be accidentally pulled.

[stasadev]

Rebased on the master.

This is an interesting PR and I'm going to continue working on it.

I think it's worth making $XDG_CONFIG_HOME/ddev optional, since DDEV has been around for a long time and there are many tutorials that depend on ~/.ddev.

I like the way it is done for git:

write to global ~/.gitconfig file rather than the repository .git/config, write to $XDG_CONFIG_HOME/git/config file if this file exists and the ~/.gitconfig file doesn’t.

So, if the user decides to move ~/.ddev to ~/.config/ddev (or other equivalent for another OS), we'll let them do it, otherwise we'll leave it as it is. From this point of view, we will not need to change all the documentation.

[danepowell]

Sounds good to me, thanks for picking this up!

[stasadev]

Changes:

• update the documentation
• remove global_config_path from ddev config global
• add ddev mutagen version command (to show the binary path and MUTAGEN_DATA_DIRECTORY)
• add global-ddev-dir to ddev version output
• use fallback value for empty $XDG_CONFIG_HOME for Linux only ($HOME/.config)
• rename MoveGlobalDdevDir to CopyGlobalDdevDir
• run mutagen length check only for Linux and macOS