Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] Docker Container keep restarting. #2454

Closed
3 tasks done
shubham-cmyk opened this issue Jul 8, 2022 · 2 comments
Closed
3 tasks done

[Bug] Docker Container keep restarting. #2454

shubham-cmyk opened this issue Jul 8, 2022 · 2 comments
Labels
type/bug This issue is a bug

Comments

@shubham-cmyk
Copy link
Contributor

shubham-cmyk commented Jul 8, 2022

Search before asking

  • I had searched in the issues and found no similar issues.

What happened

Docker container keep restarting.
馃憖 Ubuntu Local host
image

馃憖 AWS
image (5)

What you expected to happen

馃憖 Docker Container logs
image (6)

Container are not starting up in proper order.

How to reproduce

Release Version : 0.10.1

Follow up the general installation method as stated in Readme

Anything else

We are using v3 docker-compose which has removed ability of container for wait for eachother.

Version

v0.10.1

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

@shubham-cmyk shubham-cmyk added the type/bug This issue is a bug label Jul 8, 2022
@keon94
Copy link
Contributor

keon94 commented Jul 8, 2022

The ordering is technically fine (depends_on field in the compose file manages the ordering). The problem is, each container has a bootstrap process that takes some time to complete (e.g. mysql), so just because it has started it doesn't mean it's ready. In this case devlake is trying to talk to mysql before it is fully ready, and that throws an error.

With docker-compose v2 one would define healthchecks in the compose file and have the containers wait using conditions until their dependencies are healthy before boostrapping. v3 removed this feature as far as I am aware.

So our options are:

  1. Use the v2 docker-compose spec, and use healthchecks and conditions in the compose file
  2. Enhance the devlake service logic to retry DB connectivity for a while
  3. Apparently the more recent compose versions allow backwards portability as far as support for conditions go see this discussion. Maybe investigate this and see if it'll work.

@hezyin
Copy link
Contributor

hezyin commented Jul 11, 2022

@shubham-cmyk Hi Shubham, it's normal for the DevLake container to restart a couple of times while waiting for the MySQL container to boot up, but usually, it wouldn't take longer than 15 seconds. If you see the DevLake container constantly restarting while the MySQL container is up and running, then there's something wrong with the connection between the DevLake container and the MySQL container. I'd suggest that you double-check the connection string in the .env file. Make sure you use mysql:3306 instead of localhost:3306 in the connection string. Let me know if the problem persists. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/bug This issue is a bug
Projects
None yet
Development

No branches or pull requests

3 participants