Work around database storage problems on Docker for Windows/Mac/Docker Toolbox

[cweagans]

What happened (or feature request):

MariaDB in a Debian-based container on Docker for Windows with a host-mounted volume for /var/lib/mysql fails to start (has trouble writing the data files to disk on the network mount).

MariaDB is also failing in new and fun ways with Docker Toolbox, which is the path toward supporting Windows 10 Home + older Macs.

To get around this, I propose that we do the following:

• Stop using host mounts for /var/lib/mysql and use Docker volumes instead
• When a database container is stopped, dump the database to a host mount (mysqldump or similar)
• When a database container is started, if the dump file is present, import it and delete it

The last two points would be sort of optional, but very convenient. We could also not include them by default and document how to set up that behavior in the app's hooks.

This sidesteps the issue of keeping the mysql data directory on a network mount, and also allows for persisting the database contents on the host. It also allows us to not dive into mariadb's source and figure out complex problems that we don't have the time to fully understand.

What you expected to happen:

How to reproduce this:

Version: Please include the output of ddev version, docker version and the project's .ddev/config.yaml.

Anything else do we need to know:

Related source links or issues:

Please use a complexity rating of 1-5 (5 is high) for a feature request. (High complexity implies more PR planning)

[rfay]

This definitely has potential. Shortcoming: A docker reset destroys all databases for all projects. If things have been stopped/rm-ed then this is OK, and probably not much lost anyway.

We might be able to just use the tar-up-mysql approach since we use INNODB_FILE_PER_TABLE, and it would be much quicker than mysql import.

[rfay]

Note that Mariadb 10.2 has this problem even on macOS. See https://jira.mariadb.org/browse/MDEV-16015, a never-ending set of problems with Mariadb. There's a good wrapup in wodby/docker4drupal#244 (comment) (solution suggested is same as here) and MariaDB/mariadb-docker#95

[rfay]

Note that the proposal here was actually implemented in #349 as a competing approach to the direct-mounting of the db. @beeradb 's comment at that time was

We've decided to go with host mounting for volumes. The most compelling reason we did this was that we're always willing to bet against docker stability, and we'd like to maintain the ability for people to reset their local docker setup if needed. We'll still persist data to the host via #337

And that betting against docker stability is a pretty reasonable thing. If we go with volumes, a docker event can destroy everything. However, @cweagans suggests in this issue a backup dump, perhaps with xtrabackup.

[rfay]

I'm taking an initial look at this to make sure I understand the territory. This may just be a short experimental phase to determine where we want to go with it.

[rfay]

Sadly, docker volumes don't support anything but root mount permissions, moby/moby#2259 and docker/compose#3270

There is an interesting workaround (Run a temp container first that chowns the voume, and make the real container depend on that one (which exits right after chowning).

[rfay]

We might be able to switch back to the start-as-root-and-sudo-to-correct-user approach, which uses an environment variable to specify the correct UID within the container.

Or why not just add sudo into the mariadb container like we do in the web container, and let the startup script sudo the perms?