Document suggested workaround for no connections until DB init complete

[edmorley]

When using docker-compose, even if one service is marked as depending on the mysql service, the first service is started before the initial DB init is complete - and it is then unable to initially connect to the mysql service, causing errors:
https://github.com/docker-library/docs/tree/master/mysql#no-connections-until-mysql-init-completes

It would be good to add to the documentation at that link with a couple of suggested workarounds, eg using mysqladmin ping (example) or nc (for when mysqladmin is not available; slightly over-complex example) - to save everyone from having to reinvent the wheel :-)

[tianon]

+1, seems sane to me to add examples of how to accomplish this reasonably

[rothnic]

Take this back, not working now... Leaving things below for reference.

I just worked around this, then found this issue. My use case was for running tests, so I did this:

FROM mysql

ENV MYSQL_ROOT_PASSWORD=testing
ENV MYSQL_DATABASE=test

COPY entrypoint.sh /entrypoint.sh
RUN /entrypoint.sh mysqld

EXPOSE 3306
CMD ["mysqld"]

Where entrypoint.sh is the same as before, except has exit 0 after this statement, which avoids running the mysqld command I gave it. The problem is that it is dependent on mysqld to be provided, so the script needs to be reworked to allow simply doing RUN /entrypoint.sh in your own dockerfile, then you are good to go.

I'm sure there is a better solution, but the referenced ones seem hacky as well.

[mohamnag]

Have the same problem with a docker compose file which includes a mysql container. Other containers are much faster to start and before mysql is even initialised, they fail because DB is not accessible.

[yosifkit]

@mohamnag, you have a few options, start mysql first and then do compose up; write your application to wait for mysql to come up like wordpress does; specify a bind mount for the mysql database in the yaml file and once initialized, it will be fast to start every time.

[brycereynolds]

Like @yosifkit said you can start mysql first. A clean way to do this is to use a separate docker-compose for dbs and use the external link feature of compose to link to the db container (https://docs.docker.com/compose/yml/#external-links).

[mohamnag]

as I have only one DB container, for now I just start that one and then do
docker-compose up.

On Wed, 12 Aug 2015 at 04:32 Bryce Reynolds notifications@github.com
wrote:

Like @yosifkit https://github.com/yosifkit said you can start mysql
first. A clean way to do this is to use a separate docker-compose for dbs
and use the external link feature of compose to link to the db container (
https://docs.docker.com/compose/yml/#external-links).

—
Reply to this email directly or view it on GitHub
#81 (comment)
.

[sanderboom]

Big usability impairment for docker-compose IMO! A bit disappointed as this common use-case needs workarounds. If it can't be fixed, I agree documentation is needed.

@yosifkit can you elaborate?

specify a bind mount for the mysql database in the yaml file and once initialized

And, would this be a clean workaround? From MariaDB:

The my_wait_socket script blocks until the MariaDB Enterprise Server process has created the socket file. This is useful when you are starting a new container and don't want to try using the container until the server inside is running and is ready to accept connections.

https://mariadb.com/kb/en/mariadb-enterprise/mariadb-enterprise-in-docker/#my_wait_socket

[ltangvald]

As I understand it, compose will report a container as ready as soon as it starts (so when the entrypoint script starts executing), so the MariaDB workaround would still need to be added to the container that depends on the database, similar to other ways to check if the database is up and running. Some way for the database (or any container) to tell Docker they're "ready" would be nice :)

[sanderboom]

Some way for the database (or any container) to tell Docker they're "ready" would be nice :)

True. See also: docker/compose#374

[yosifkit]

Something like the following:

app:
  image: wordpress
  links:
    - db:mysql
  ports:
    - 8080:80

db:
  image: mysql
  environment:
    MYSQL_ROOT_PASSWORD: example
# this is the important part:
  volumes:
    - ./mysql/:/var/lib/mysql

Then start up just the mysql container

$ docker-compose up db
........... wait for it to initialize and be ready for network connections:
db_1 | 2015-10-20 23:00:11 1 [Note] Server hostname (bind-address): '*'; port: 3306
db_1 | 2015-10-20 23:00:11 1 [Note] IPv6 is available.
db_1 | 2015-10-20 23:00:11 1 [Note]   - '::' resolves to '::';
db_1 | 2015-10-20 23:00:11 1 [Note] Server socket created on IP: '::'.
db_1 | 2015-10-20 23:00:11 1 [Warning] 'proxies_priv' entry '@ root@04463714990e' ignored in --skip-name-resolve mode.
db_1 | 2015-10-20 23:00:11 1 [Note] Event Scheduler: Loaded 0 events
db_1 | 2015-10-20 23:00:11 1 [Note] mysqld: ready for connections.
db_1 | Version: '5.6.27'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306  MySQL Community Server (GPL)
^CGracefully stopping... (press Ctrl+C again to force)
Stopping wordpress_db_1... done

Now docker-compose up should just work. The database files will be stored in ./mysql/ (owned by 999:999). You can stop and even delete your docker-compose containers (docker-compose rm). The mysql container will be quite quick since the database initialization is already done; it only takes about 3 seconds on my machine.

So this works when both are stopped and the app takes longer to start than MySQL. I think the best solution is to create your apps with tolerance on waiting for the database to be available and a configurable timeout for failure. It should be similar to the way that you make one "microservice" tolerant of another "microservice" that is unresponsive.

Technically this has always been an issue in database connected applications, but we have usually either already had a database running (on our host for dev usage), or had the system admin spin one up beforehand.

[ltangvald]

Adding something like the nc-example plus maybe the suggestion of starting the database container separately to docs would be good, I think.

[ephrin]

+1 for nc ping like initialization check.
Can't find any public interface definition for such case but it should exist for sure. As relying on pid or socket file does not mean the process is initialized and ready to communicate.
The way to implement decision logic of state ready to do communication by app stack itself not so trivial but necessary in many cases for simple and reliable apps/services.

[saulshanabrook]

I didn't use nc because it is not available as often as the socket file. For example, alpine doesn't have nc installed by default.

[nwinkler]

I've started to use https://github.com/vishnubob/wait-for-it for this. Pulling this in as a Git subtree is easy, and I can then map it into the containers that need to wait for the DB to be up.

In the container, I run a script as the command, and in the script I use the following as the first item:

/usr/local/wait-for-it/wait-for-it.sh db:3306

Works great for me!

[peacepassion]

@nwinkler I used wait-for-it.sh but not successful at first compose. After the first failure, the following compose up would be OK. The possible reason is there is a init process for MySql first run. The log is the following.

Before this init process is done, although the port is ready, the connection to the db would be fail.
In my python web app, it would produce following error.

[micahjsmith]

I had success scanning the docker logs:

grep -m 2 "mysqld: ready for connections." <(docker logs --follow "$MYSQL_CONTAINER_NAME")

[tianon]

@micahjsmith interesting solution! I would caution that it's going to cause problems if you ever use docker restart (or your container is restarted on its own), since the logs will still contain the old output

[tianon]

See also the discussion in #2 -- I'm going to close, since there isn't really a generic way we can solve this cleanly, but I think the existing discussion will serve as a good reference point for future travelers!

[edmorley]

Hi! This issue was about collecting together the various options and documenting them in the README (which would be more discoverable than having to find these issues) - so still seems actionable? Failing that, perhaps even linking to those issues from the README section linked in the OP might be a good stop-gap?

[tianon]

I think my main concern with doing anything here is that this is really a generic Docker issue, not something specific to MySQL -- any container which needs to talk to another container (or relies on another container) requires similar consideration and care, so it feels wrong to document this in the context of the MySQL image. 😕

[almereyda]

Redundance is good for humans, despite the once and only once pattern for software. Reopening and only closing if the README got updated a little with pointers to any case sounds good to me.

…

On 28 December 2017 at 18:54, Tianon Gravi ***@***.***> wrote: I think my main concern with doing anything here is that this is really a generic Docker issue, not something specific to MySQL -- any container which needs to talk to another container (or relies on another container) requires similar consideration and care, so it feels wrong to document this in the context of the MySQL image. 😕 — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#81 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABka_FMrUsMMYxh9Lq4CyQRQvj-AdIq-ks5tE9XZgaJpZM4FH8M7> .

[tianon]

Yeah, I suppose having a tad bit more than what we had there seems reasonable, see docker-library/docs#1106.