Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for developing the rails app within Docker #726

Merged
merged 3 commits into from Aug 1, 2018

Conversation

jgondron
Copy link
Contributor

Add support for developing the rails app within Docker

75f89dd

Some notes about the rationale behind some of the decisions made in implementing this:

  • Initially got a version of this working with rails in a container with a mounted volume for the app code, but it's really really slow. Appears to be a known problem when using a mounted volume on OSX https://forums.docker.com/t/file-access-in-mounted-volumes-extremely-slow-cpu-bound/8076/276. Added docker-sync configuration and instructions on how to install/use.
  • Building the image will copy the full application code into /project_root within the container instead of /app. This is in order to disambiguate this from the app directory within the root of the project. Most of the docs for docker-sync will reference /app, so just be aware of this
  • Added sync excludes for common unneeded directories that may exist in dev workdirs. If you do begin using this, it would be a good idea to clean out any temp files you have in your directory to speed up build/syncs
  • There are some shenanigans I have to play with the bundled gems and the Gemfile. The rails container will bundle gems into a /bundle directory. This is to preserve any changes bundler will make to the Gemfile.lock, so that we can copy it back to the host once sync is started. I'm running an additional command at the end of the Dockerfile to copy of the lock file back to project_root in the case when someone is not using docker-sync
  • Had to add a USER env to the compose to reproduce the "developer is an admin" behavior that we all expect
  • Added install of fits (and the java runtime) to the rails image so that rake curatend:ci would run correctly
  • The rails_entry script will additionally create the test db for you to simplify running tests in the container. We may want to remove this in the future and separate the test scripts/compose/etc from dev
  • The rails app is not very fault tolerant if, at start time, mysql isn't ready. Initially I added a healthcheck on mariadb and a rails depends_on mysql healthy condition, thinking this should prevent rails from starting until mysql is healthy. Unfortunately, the healthy dependency condition is no longer available in v3 of docker compose. Instead, this required copying the wait-for-it script referenced in Can this work with v3 of docker compose? peter-evans/docker-compose-healthcheck#3 and adding a rails entry script that will handle all the things associated with first starting the rails app, including waiting for mysql to start. Left the healthy script there since it may still prove useful
  • Broke out the readme into separate readme files now that there is an entirely different set of instructions when using Docker and Docker-sync. README.md will continue to give instruction on how to install/run locally on a host. README_DOCKER.md will contain instructions on how to do these things with Docker
  • Finally, I just want to again note that this only solves the problem of using Docker in development. We need to do things differently if we want to use Docker for CI and other deploys. There are pieces of this that can be reused, but how we separate those out into a common base dockerfile/compose files would need to be discussed

Ideally, adding this would have required making no changes to existing app code, but I did have to make several changes:

  • Running rails app under compose needs to use network names for external services, ie: specs and configs can't assume things are on localhost. Changed solr/fedora/database configs to use env if available, but fallback to localhost if not there to continue supporting development on the host without Docker (such as in the Travis ci atm)
  • Had to add config.allow_http_connections_when_no_cassette to spec_support. The config was already ignoring localhost, which worked when developing locally on a host. In docker this won't work because it will make calls to the "jetty" and "mysql" hosts and will fail because it couldn't find a cassette that matches. Likewise, changed the id_service_spec to just match on path and method for the cassette so that it would successfully match when a spec made the call to jetty.
  • Added a byebug initializer so that we can debug remotely, since this is technically a remote machine
  • Added .docker-sync to gitignore for cases when someone runs the sync as a daemon
  • Now have multiple Dockerfiles, so had to do a little clean up of the file locations. Note: This docker-compose is still for development only. We'll need to break this out later if we ever decide to use this for ci/staging/etc

Justin Gondron added 2 commits July 27, 2018 14:38
Some notes about the rationale behind some of the decisions made in implementing this:
  - Initially got a version of this working with rails in a container with a mounted volume for the app code, but it's really really slow. Appears to be a known problem when using a mounted volume on OSX https://forums.docker.com/t/file-access-in-mounted-volumes-extremely-slow-cpu-bound/8076/276. Added docker-sync configuration and instructions on how to install/use.
  - Building the image will copy the full application code into /project_root within the container instead of /app. This is in order to disambiguate this from the app directory within the root of the project. Most of the docs for docker-sync will reference /app, so just be aware of this
  - Added sync excludes for common unneeded directories that may exist in dev workdirs. If you do begin using this, it would be a good idea to clean out any temp files you have in your directory to speed up build/syncs
  - There are some shenanigans I have to play with the bundled gems and the Gemfile. The rails container will bundle gems into a /bundle directory. This is to preserve any changes bundler will make to the Gemfile.lock, so that we can copy it back to the host once sync is started. I'm running an additional command at the end of the Dockerfile to copy of the lock file back to project_root in the case when someone is not using docker-sync
  - Had to add a USER env to the compose to reproduce the "developer is an admin" behavior that we all expect
  - Added install of fits (and the java runtime) to the rails image so that rake curatend:ci would run correctly
  - The rails_entry script will additionally create the test db for you to simplify running tests in the container. We may want to remove this in the future and separate the test scripts/compose/etc from dev
  - The rails app is not very fault tolerant if, at start time, mysql isn't ready. Initially I added a healthcheck on mariadb and a rails depends_on mysql healthy condition, thinking this should prevent rails from starting until mysql is healthy. Unfortunately, the healthy dependency condition is no longer available in v3 of docker compose. Instead, this required copying the wait-for-it script referenced in peter-evans/docker-compose-healthcheck#3 and adding a rails entry script that will handle all the things associated with first starting the rails app, including waiting for mysql to start. Left the healthy script there since it may still prove useful
  - Broke out the readme into separate readme files now that there is an entirely different set of instructions when using Docker and Docker-sync. README.md will continue to give instruction on how to install/run locally on a host. README_DOCKER.md will contain instructions on how to do these things with Docker
  - Finally, I just want to again note that this only solves the problem of using Docker in development. We need to do things differently if we want to use Docker for CI and other deploys. There are pieces of this that can be reused, but how we separate those out into a common base dockerfile/compose files would need to be discussed

Ideally, adding this would have required making no changes to existing app code, but I did have to make several changes:
  - Running rails app under compose needs to use network names for external services, ie: specs and configs can't assume things are on localhost. Changed solr/fedora/database configs to use env if available, but fallback to localhost if not there to continue supporting development on the host without Docker (such as in the Travis ci atm)
  - Had to add config.allow_http_connections_when_no_cassette to spec_support. The config was already ignoring localhost, which worked when developing locally on a host. In docker this won't work because it will make calls to the "jetty" and "mysql" hosts and will fail because it couldn't find a cassette that matches. Likewise, changed the id_service_spec to just match on path and method for the cassette so that it would successfully match when a spec made the call to jetty.
  - Added a byebug initializer so that we can debug remotely, since this is technically a remote machine
  - Added .docker-sync to gitignore for cases when someone runs the sync as a daemon
  - Now have multiple Dockerfiles, so had to do a little clean up of the file locations. Note: This docker-compose is still for development only. We'll need to break this out later if we ever decide to use this for ci/staging/etc
… jetty image assumed we only had one Dockerfile in root.
@@ -0,0 +1,2 @@
* text=auto
*.sh text eol=lf
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this file bothers anyone, I can remove. This would just be a nice to have for running this in a container env on Windows. Without it, git on windows will convert to crlf and break the scripts running inside the containers.

Copy link
Contributor

@jmille79 jmille79 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Everything has been working consistently for me for the past two days.

@jgondron jgondron merged commit 0d69a8d into master Aug 1, 2018
@jeremyf jeremyf deleted the docker-compose-rails-with-sync branch August 20, 2018 15:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants