Implement releases using Docker

[andreskytt]

Implement a release process that results in a bunch of docker containers encapsulating the following parts of the system:

• load balancer
• Static web
• FoxAPI
• LoginAPI
• Mock LDAP

[peterjasc]

Need to access config/ folder inside the Dockerfile that is in static/ folder.

The host directory is, by its nature, host-dependent. For this reason, you can’t mount a host directory from Dockerfile because built images should be portable. A host directory wouldn’t be available on all potential hosts.

(ie, VOLUME ~/git/fox/:/containerdir/ is illegal)

For security reasons, you can not do ADD ../aa /static either (add parent directories -- since they are outside of the build context)

[peterjasc]

One possible solution was (which necessitates .dockerignore in /fox) this: moby/moby#2745

I successfully worked around this issue using volumes. Example directory structure:

web/

• Dockerfile -> ADD . /usr/src/app/
• (code)

webstatic/

• Dockerfile -> ADD . /usr/src/app/subdir
• (code)

compose file excerpt:

web:
build: ./web/

webstatic:
build: ./webstatic/
volumes_from:
- web

Then in the webstatic code I can access the code from the web container. No need for ADD ../web/ .. in webstatic.

[peterjasc]

This is not the best solution, since it uses data only containers (Docker Compose file ver 2 seems to have deprecated these with named volumes). A better idea seems to give the dockerfile (in static/) the project's root as the build context (so it could access both static/ and config/).

The implementation of this concept can be seen at 4528de3

[peterjasc]

Links and "expose" (similar to "ports" command, but does not publish ports to host machine) can be considered deprecated features since Docker 1.9 (and should not be used, when writing Docker Compose files of ver 2). Networks should be used instead.

However:

The default docker0 bridge network supports the use of port mapping and docker run --link to allow communications between containers in the docker0 network. These techniques are cumbersome to set up and prone to error. While they are still available to you as techniques, it is better to avoid them and define your own bridge networks instead.

Implementation of user-defined bridge networks and loadbalancers:
f0feb24
Loadbalancer configs:
895f239

https://docs.docker.com/engine/userguide/networking/dockernetworks/
https://docs.docker.com/compose/compose-file/#/expose

[peterjasc]

Building go files inside a Docker container with go build means that GOPATH needs to point to the project root. This caused some problems in figuring out how to get the build to use a local package's external dependencies located in out project (for example, local pkg "loginservice" is dependant on the following external dependencies located in out project -- "authn", "authz", "login" and "util"), without copying the whole project into every container.

The solution is quite simple: we just build a similar directory structure inside the container and only copy the files we need, with a small caveat -- we will use the vendor directory for local pkg's local dependencies.

Go 1.6 includes support for using local copies of external dependencies to satisfy imports of those dependencies, often referred to as vendoring.

For example, to build Loginservice, we copy authn, authz and util into the vendor folder, instead of directly into the source directory (a small difference, but it makes it clear that these are the external dependencies of pkg login).

├── src
...
    ├── login
        └── vendor
        │ └── authn
        │ └── authz
        │ └── util
        └── loginservice

As of July 2016, the "dependency management"/"go vendoring" tools are not mature enough for use in production, so, in order to get all the dependencies required for a build, we will have to copy these manually from the host's (from the point of view of the Docker container) source directory into the container's vendor directory and use go get -d to get the other external dependencies (such as github.com/fernet/fernet-go).

---

For reference:
https://golang.org/cmd/go/#hdr-Vendor_Directories

[peterjasc]

Make static dockerfile buildable 1cdede4

Remove volume, since it hides the actual folder from Docker container 993532a

Replace compass with libsass to not have to get ruby in container
fad0c7b

Copy properties into the right directory and update node dependencies
3cb8c73

[peterjasc]

For haproxy logging, we used rsyslog and mounted /dev/log on the haproxy containers, which means we can read the log at /var/log/syslog

References:
dockerfile/haproxy#3 (comment)
https://youroldmanspath.blogspot.com.ee/2015/12/logging-and-haproxy-docker-container.html

[peterjasc]

Create a network so that static_web proxies go through load balancers
(Previously, static_web used Fox Service and Login Service directly.)
7351e93

[peterjasc]

set the correct status check address for balancers 3468583