(WIP) Machine declaration

[kunalkushwaha]

This is draft code for review.

With reference to Issues #773, #2303, This code implements a very basic implementation of docker-machine apply command.

With this path, One or more machines can be created at one go with docker-machine apply command.

• This simply provides alternative to multiple docker-machine create command.
• supports swarm.
• Does not provide separate mechanism to load secrets from.
• serially create machines.

Need your feedback and start discussion again, so these issues #773 & #2303 can be focused.

A sample yaml config file as below it understands.

---
- name: test-host0
  driver: virtualbox
  driveroptions:
    image-name: "boot2docker"
  engineoptions:
    env:
    - HTTP_PROXY="http://example.com:8080"
    - HTTPS_PROXY="http://example.com:8080"
    - NO_PROXY="192.168.99.101"
    arbitraryflags:
    - cluster-advertise=eth1:3376
    - cluster-store=consul://192.168.99.101:8500
  swarmoptions:
    isswarm: true
    master: true
    discovery: consul://192.168.99.101:8500
- name: test-host1
  driver: virtualbox
  driveroptions:
    image-name: "boot2docker"
  engineoptions:
    env:
    - HTTP_PROXY="http://example.com:8080"
    - HTTPS_PROXY="http://example.com:8080"
    - NO_PROXY="192.168.99.101"
    arbitraryflags:
    - cluster-advertise=eth1:3376
    - cluster-store=consul://192.168.99.101:8500
  swarmoptions:
    isswarm: true
    master: true
    discovery: consul://192.168.99.101:8500

[nathanleclaire]

I'm definitely interested. We can't merge it in right away for a variety of reasons, but maybe we can work together towards getting this in an experimental state and then refining it for 0.6.0.

[kunalkushwaha]

Yes, This PR is not intended to be merged.

I intentionally kept it very basic to start the discussion. Need feedback for same.
On following things I would like to have clarity.

• Scope of this docker-machine apply command.
  • Since Proposal: Machine Declaration #773, doesn't conclude any points like.
  • Creation of docker-host with swarm, just like docker-machine create
  • Creation of swarm host with application deployment (linking containers etc) , Something like machine + compose.
• Format of config file.
• Support for machine versions. ( Upgrade/downgrade to different machine state, depending of config versions)
  • Essentially, Modify/recreate hosts again.
• How to handle secrets:
  • A plugin based model to support interface for getting secrets from third party software like Vault, Keywiz etc

[nathanleclaire]

What's the license on the vendored YAML parser?

Scope of this docker-machine apply command

My feeling is it should start limited in scope and grow outwardly from there if needed. To that extent, I think it should focus on defining specifications for machines to be created, and then updating the supported configuration (engine labels, swarm, etc.) via SSH if they exist already. Modification of driver parameters after create time should not be permitted. Therefore, I don't want to include any support for Compose or running containers in the first versions.

Format of config file.

Ideally, all supported options in the file should be as close as possible to their CLI flag counterparts (using struct tag like yaml:"engine" for EngineOptions, for instance). So, your given example might be:

---
- name: test-host0
  driver: virtualbox
  driveropts:
    boot2docker_url: "file:///tmp/boot2docker.iso"
  engineoptions:
    env:
    - HTTP_PROXY="http://example.com:8080"
    - HTTPS_PROXY="http://example.com:8080"
    - NO_PROXY="192.168.99.101"
    opts:
    - cluster-advertise=eth1:3376
    - cluster-store=consul://192.168.99.101:8500
  swarm:
    # isswarm is implied by presence of this key
    master: true
    discovery: consul://192.168.99.101:8500
- name: test-host1
  driver: virtualbox
  driveropts:
    boot2docker_url: "file:///tmp/boot2docker.iso"
  engine:
    env:
    - HTTP_PROXY="http://example.com:8080"
    - HTTPS_PROXY="http://example.com:8080"
    - NO_PROXY="192.168.99.101"
    opts:
    - cluster-advertise=eth1:2376
    - cluster-store=consul://192.168.99.101:8500
  swarm:
    discovery: consul://192.168.99.101:8500

It's a bit tricky since I want to be as consistent with compose as we can, so e.g. boot2docker-url becomes boot2docker_url.

Support for machine versions. ( Upgrade/downgrade to different machine state, depending of config versions)

I'm not sure what you mean by this. Care to elaborate?

[kunalkushwaha]

What's the license on the vendored YAML parser?

https://github.com/cloudfoundry-incubator/candiedyaml is Apache 2.0 Licence. The reason why I chose, is usage of this parser in libcompose

Modification of driver parameters after create time should not be permitted. Therefore, I don't want to include any support for Compose or running containers in the first versions.

I agree. Such features can be added later.

updating the supported configuration (engine labels, swarm, etc.) via SSH if they exist already

Can we use DockerAPI's. Currently working on a patch that will replace swarm provisioning using Docker API instead of go template based docker commands.

It's a bit tricky since I want to be as consistent with compose as we can, so e.g. boot2docker-url becomes boot2docker_url

Fine, I will also go through both and come with config file specs.

Support for machine versions. ( Upgrade/downgrade to different machine state, depending of config versions)

Here idea was something similar to terraform plan and terraform apply https://terraform.io/intro/getting-started/build.html

Before applying changes to host, yaml file can be tested (lint & vet) for possible errors and on success, It can be translated to machine-readable config, tagged with version. (Default action can be auto assigning incremental versions)

Benefit of this approach are

• Testing host with different configs will be easy to manage.
• In case of accidental changes, rollback will be easy.
• Also, detecting errors like change in driver-options can be easily done.

[nathanleclaire]

Can we use DockerAPI's. Currently working on a patch that will replace swarm provisioning using Docker API instead of go template based docker commands.

Oh, awesome. You should make sure to take a look at master. We have vendored the dockerclient module and introduced a way to get a dockerclient for a machine.

[nathanleclaire]

Before applying changes to host, yaml file can be tested (lint & vet) for possible errors and on success, It can be translated to machine-readable config, tagged with version. (Default action can be auto assigning incremental versions)

I do agree this type of thing could potentially be very useful in the future, however I think for the first versions we should keep scope small and just have something which brings the system to the stated configuration.

To get there we already have to:

• Make provisioning idempotent(ish). For example, it can't handle already-running Swarm containers today. Likewise it has no notion of "the certs already exist and are valid, so skip generating and sending those".
• Check if machines exist with the desired names, then bring them up if they don't.

[kunalkushwaha]

We have vendored the dockerclient module and introduced a way to get a dockerclient for a machine.

Thanks for update, Will adapt DockerHost interface.

I think for the first versions we should keep scope small and just have something which brings the system to the stated configuration.

Fine, lets keep first version simple.

And one doubt about execution of host creation.

• Should host creation be done in parallel or serial?
• Have not checked log logic yet, but In case if it requires to be done in parallel, how to collect and show logs in case of error.

[kunalkushwaha]

The PR is updated with changes related to yaml file for review.
@nathanleclaire Please take a look at it. Below is dump of config file with tags.

For driveroptions, changes are done at client side (machine) not required in drivers. So drivers can interpret options, same way as they do currently.

• drivername will be automatically prepend to its tags under driveroptions, and replace _ with - to match commandline options, which while passing to drivers, it is same as commandline options.
• For engineoptions and swarmoptions structure tags are used.

---
- name: localvm3
  driver: virtualbox
  storage_path: ""
  tls_ca_cert: ""
  tls_ca_key: ""
  tls_client_cert: ""
  tls_client_key: ""
  github_api_token: ""
  native_ssh: false
  driveroptions:
    boot2docker_url: "file:///tmp/boot2docker.iso"
    disk_size: 50000
    host_dns_resolver: true
  engineoptions:
    opt:
    - cluster-advertise=eth1:3376
    - cluster-store=consul://192.168.99.101:8500
    graph_dir: ""
    env:
    - HTTP_PROXY="http://example.com:8080"
    - HTTPS_PROXY="http://example.com:8080"
    - NO_PROXY="192.168.99.101"
    ipv6: false
    insecure_registry: []
    label: []
    selinux_enabled: false
    registry_mirror: []
    install_url: ""
  swarmoptions:
    addr: ""
    discovery: consul://192.168.99.101:8500
    master: true
    host: ""
    image: ""
    strategy: ""
    opt: []

[kunalkushwaha]

Current state of PR.

• Creates machine, if not present.
• If machine already exists and stopped,
  • Skip Cert keys generation and copy to machine.
  • it starts the machine and start swarm-agent-master & swarm-agent containers.

Need code review.

NOTE: provisioning fixed for boot2docker only.

[nathanleclaire]

Just a quick touch base @kunalkushwaha : Because it's such a big feature, I think we are going to have to hold off on adding this until we do some more project planning and defining how it fits with Machine's future (as well as the future of other Docker tools). We'll keep you posted.

[kunalkushwaha]

@nathanleclaire Thanks for update :)