add 3rd party docker extensions

[kacole2]

this will allow 3rd parties to install their docker extensions on the
host as a part of the host installation/creation process.

Signed-off-by: Kendrick Coleman kendrickcoleman@gmail.com

[kacole2]

Extensions for Docker Machine

This PR is to address Proposal / Issue #1765

Premise

Docker Machine gets you a "docker ready" host, but what about everything else that integrates with Docker Engine? Docker Engine 1.8 released the pluggable architecture for 3rd party network and storage extensions. However, most of these require installation processes that must be done after the host is provisioned. The Docker Machine Extensibility feature is an interface for specifying the installation of Docker Engine Plugins to install during the host deployment.

Extensions Supported:

• REXray : stable
• Weave : stable
• Generic - install services through a JSON file without customization

Summary

Extensions for Docker Machine is a pluggable architecture to install Docker Engine Plugins as an additional command line argument.

docker-machine -D create --driver amazonec2 --amazonec2-access-key mykey --amazonec2-secret-key mysecretkey --amazonec2-vpc-id myvpcid --amazonec2-zone b --extension /Users/kcoleman/Desktop/extensions.json dev

The name of the .json file is arbitrary. This allows the creation of multiple .json files for different machines provisioned through Docker Machine. At this time, .json files are the only supported file types.

The .json file supports the following structure and fields:

{
    "extensions": [
        {
            "extension-name": {
                "version": "latest",
                "env": {
                    "any environment": "variables"
                },
                "params": {
                    "specific key values": "for extension customization"
                },
                "copy": {
                    "source path/file": "destination path/file"
                },
                "files": {
                    "filename for extension customization": {
                        "source": "source path/file",
                        "destination": "destination path/file"
                    }
                },
                "run": [
                    "commands",
                    "to",
                    "run"
                ],
                "validOS": [
                    "OSnames",
                    "ubuntu"
                ]
            }
        }
    ]
}

A generic installation of a service using only commands lines:

{
    "extensions": [
        {
            "generic": {
                "envs": {},
                "copy": {
                    "/Users/kcoleman/Desktop/rexconfig.yaml": "~/.rexray/config.yaml"
                },
                "run": [
                    "curl -sSL https://dl.bintray.com/emccode/rexray/install | sh",
                    "sudo service rexray start"
                ],
                "validOS": [
                    "ubuntu",
                    "debian",
                    "centos",
                    "redhat"
                ]
            }
        }
    ]
}

A simple example of installing a service (Weave as an example):

//weave-node-01 (ie. 192.168.10.101)
{
    "extensions": [
        {
            "weave": {}
        }
    ]
}

//weave-node-02 
{
    "extensions": [
        {
            "weave": {
                "params": {
                    "peer": "192.168.10.101"
                }
            }
        }
    ]
}

a more complex example with extensions running in priority:

{
    "extensions": [
        {
            "generic": {
                "copy": {
                    "/Users/kcoleman/Desktop/rexconfig.yaml": "/etc/rexray/config_backup1.yaml"
                }
            }
        },
        {
            "weave": {}
        },
        {
            "rexray": {
                "version": "latest",
                "files": {
                    "config.yaml": {
                        "source": "/Users/kcoleman/Desktop/rexconfig.yaml",
                        "destination": "/etc/rexray/config.yaml"
                    }
                }
            }
        },
        {
            "generic": {
                "copy": {
                    "/Users/kcoleman/Desktop/rexconfig2.yaml": "/etc/rexray/config_backup3.yaml"
                }
            }
        }
    ]
}

Deeper Dive

The .json file is read from a local resource into memory.

The name of the extension is checked to see if a registered extension exists. If there is a matching registered extension, the paramters in the .json file are passed on.

• version will set the value from nil to a string in the extInfo struct. If it's not specified and the value nil is passed along, the extension's .go file will have a default or latest version specified. Versioning allows consistent deployment of the extension in the case that a specific docker-engine version is being specified in the docker machine command. Each extension is responsible for the implementation of installing different versions.
• env is a set of key:value pairs that can call the utils.go function appendEnvFile to set environment variables on the host.
• params is any set of key:value pairs that an extension can use for customization. These pairs are passed on to the extension's interface where the extension can use them any way it wishes. Such use cases include commands to run or local variables for configuration.
• copy is a key:value pair that will take the source and destination and transfer the files using docker-machine scp. This takes no priority and files can be transferred at will when called.
• files is a nested set of key:value pairs for the transfering of files using the docker-machine scp command. This function is used for customizing extensions. The name specified in the .json file must match in the function called. This allows you to transfer files at any time in your code. This gives you the freedom and flexibiltiy to transfer files when a function may need them. Files can be transferred from the local system to remote systems or between remote systems. Use cases include configuration files or binaries to be copied. There is a generic function in utils.go that will take the source and destination keys to transfer files.
• run is an array of commands that will be run in sequential order. These are piped into the provisioner.SSHCommand().
• validOS is an array of supported Operating Systems to check. This also allows your to run commands based on the type of OS that is detected from the .json file.

Before the installation begins, the .json file is parsed to make sure there is valid JSON. This is denoted in the host.go file.

if h.HostOptions.ExtensionOptions.File != "" {
    if err := extension.ParseExtensionFile(*h.HostOptions.ExtensionOptions); err != nil {
      return err
    }
  }

The installation of any extension is performed as the final step during the docker-machine configuration process. This is denoted in the host.go file. It checks to see if the --extension flag isn't empty:

if h.HostOptions.ExtensionOptions.File != "" {
      if err := extension.ExtensionInstall(*h.HostOptions.ExtensionOptions, provisioner); err != nil {
        return err
      }
    }

[nathanleclaire]

Hi @kacole2 ,

I appreciate the contribution but I still don't quite understand the use case and due to the added complexity, adding a change such as this is an inherently risky move for a variety of reasons. Can we please close this and continue the discussion at : #1765 ? I want to help your use case, but the solution has to be constructed in a general manner as we really are trying to trim down the scope of Machine and allow for more extension / flexibility in the basic mechanics.

• N

EDIT: Oops, yeah, copy-paste error on the issue.

[clintkitson]

@nathanleclaire I believe the issue is #1765 that you were referring to.

[clintkitson]

Relating to the general way of achieving extensibility. I believe @kacole2's approach here tends to be a very simple way to extend whichever functionality during the machine deployment. With priority, having the ability to set environment variables, copy files, run commands, and do so against specific OS's seems like it fits the bill for simple and generic operations.

{
    "extensions": [
        {
            "generic": {
                "envs": {},
                "copy": {
                    "/Users/kcoleman/Desktop/rexconfig.yaml": "~/.rexray/config.yaml"
                },
                "run": [
                    "curl -sSL https://dl.bintray.com/emccode/rexray/install | sh",
                    "sudo service rexray start"
                ],
                "validOS": [
                    "ubuntu",
                    "debian",
                    "centos",
                    "redhat"
                ]
            }
        }
    ]
}

[clintkitson]

@nathanleclaire There is alignment that can be made here that might make more sense. If you look at the purpose of the Dockefile in ensuring a base container is configured and ready for usage this method tends to make sense. This method or file could even be the Machinefile. It's purpose here though would be to prepare a base machine that was deployed with whichever things it needs at an OS level to be ready. If you look at the params in the Generic key above, it even has copy/run/env similar to what is achieved w a Dockerfile.

[nathanleclaire]

@clintonskitson I understand that and I am definitely in favor of declarative configuration, however such a change is not something to be taken lightly. There are so many stakeholders at play in such a decision that it is very preferable to have a discussion first and see what everyone is interested in and considers to be a robust solution. Supporting such a declarative config is easy to put in, but very hard to take out down the line if it turns out that there are considerations not addressed in the original model.

[clintkitson]

@nathanleclaire Very much so. Is there another thread better suited with the active conversation for this functionality?

[kacole2]

@nathanleclaire some other use cases (for generic functionality)

1. Security Templates. Run commands or transfer files that coincide with a company's security policy. Many enterprise shops, that I know personally, run Docker on Red Hat and must have the same applied security constraints
2. Additional Configuration. Change a hosts NTP settings or anything else.
3. Additional Configuration. Add a chef or puppet agent and have it auto-configure itself for 1 or 2.

All of these can be done with the generic extension.

[nathanleclaire]

Is there another thread better suited with the active conversation for this functionality?

@clintonskitson It's not specifically suited to plugins, but the type of feature this PR introduces ties closely to the proposal outlined here, so you might want to take a look:

#773

@kacole2 Again, I'm not that I'm against such a feature or don't see the reasons why it'd be useful, it's that this pull request is very wide-ranging in terms of features and implications for users of Machine. Additionally, I'm not interested in supporting 3rd party extensions such as Weave or Rexray directly in the core. So, I'd suggest to please research what the maintainers and others have suggested and/or proposed in the past and begin an issue discussion which would address everyone's concerns, and/or a model for pluggability that would enable you to implement such features yourself.