feature: variables for scale: id, hostname

[ioggstream]

This patch enables parametric hostnames and commands when using scale
avoiding all bash variables expansion issues.

# docker-compose scale test=4
test:
  image: busybox:latest
  hostname: test-%%id%%
  command: runme.sh %%hostname%%

From #1006

Signed-off-by: Roberto Polli robipolli@gmail.com

[seal-ss]

@ioggstream Thanks! I've tested your PR. I found out that the variables only work for the command, but not for the environment key. Is there a reason not to support the environment as well?

I have a use case to configure a container with its "outside" URL that will be sent through a Rabbit Message. I would like to configure only the environment OUTSIDE_URL like this:

duplexer:
  image: 'node:0.10'
  command: 'node /code/bin/duplexer.js'
  links:
    - db
    - mq
  expose:
    - '1968'
  volumes:
    - '../duplexer:/code'
  environment:
    - MONGO_URL=mongodb://db:27017/jobs
    - RABBIT_URL=amqp://mq:5672
    - OUTSIDE_URL=http://duplexer_%%id%%:1968

But at the moment I have to move this environment into the command to make it work:

duplexer:
  image: 'node:0.10'
  command: 'bash -c "OUTSIDE_URL=http://duplexer_%%id%%:1968 node /code/bin/duplexer.js"'
  links:
    - db
    - mq
  expose:
    - '1968'
  volumes:
    - '../duplexer:/code'
  environment:
    - MONGO_URL=mongodb://db:27017/jobs
    - RABBIT_URL=amqp://mq:5672
    # see command above - OUTSIDE_URL=http://duplexer_%%id%%:1968

What do you think?

[ioggstream]

@stefanschererseal I think it's fine.

If you look at the patch, it's very easy to enable the feature on other sections.

Anyway, I think we should get it merged before doing further development...do you agree?

[StefanScherer]

I'll try to put the variables into the environment section as well. I think doing only one PR would be easier for the maintainers merge.
If I find something useful I could send you a PR into your feature branch for a review.

[ioggstream]

@stefanschererseal check if you can just add 'environment' to the list referenced here
https://github.com/docker/compose/pull/1131/files#diff-db70ffd20201b81c947f72fea35551d2R420

If that works, then we should just add proper tests.

[Kosta-Github]

+1 for this enhancement

[ioggstream]

@aanand Do you think this feature is fine? Would you implement it in a different way?

Thx for all your great work,
R.

[aanand]

I think it's interesting. It would be good to get an idea of how many people would use it. I'm not sold on the syntax, but that's a minor thing. We'd also need to think about how it would interact with environment variable resolution, were we to implement that.

[ioggstream]

Syntax: open to your suggestions, so in case of merge the code will be ready.

Environment: shouldn't be interested, but I can investigate. Link me how you expect to implement variable resolution and which are your doubts: I'll be glad to help!

[hairyhenderson]

I'm looking forward to a feature like this being implemented.

The syntax is a little unusual, however. I think it would be more natural to allow resolution of environment variables, and simply add some "special" variables which reflect the current container. Something like ${CONTAINER_NUMBER} instead of %%id%%.

Also, I think it would make more sense to use these variables throughout the entire configuration and not just in select spots.

I should also say - the only reason I really need a feature like this is because I have a situation where I'm lacking good Service Discovery. With proper SD, hostnames, container numbers, and many other variables are simply unnecessary. So, for that reason alone perhaps this feature shouldn't be implemented at all, and users should be encouraged to look at consul, etcd, registrator, interlock, etc...

[ioggstream]

@hairyhenderson thanks for your feedback! I'll show my points: I am really interested in your feedback.

Syntax: I made it weird to avoid messing with environment. I'm open to suggestions. Intended goals:

• separate environment implementation from scale implementation;
• avoid customizing entrypoints (eg. for reading parameters) and reduce the number of images.

Variables for every parameter: it's ok.

Service Discovery: you're right in many cases but:

• they are not always a necessity (eg. on your laptop ;));
• when they are needed, you're probably using kubernetes & co and not compose
• you may need to distinguish various compose services hosts (eg. the --server-id argument of mysql)

Let me know if it does make sense for you + Peace,
R.

[hairyhenderson]

@ioggstream - thanks for the clarification!

Good points about SD - I hadn't thought of the MySQL case (I haven't used MySQL in years ;)).

From my perspective, I don't really care a lot about syntax. However, this is a pretty major feature and in the interest of new users I think this should be something unsurprising and easy to stumble on.

I suppose I really have two separate issues - one is naming - I really think it should be container_number and not id. More specific that way.

The other issue is the syntax to surround variable names with. Barring sh syntax ($FOO or ${FOO}), something like {{foo}} might be a good solution. It looks sort of like the Go template syntax, which is already used in docker and docker-machine's inspect commands. And it also allows room for expansion...

I dunno, I'm really just throwing out ideas... But, like I said before, I really hope something like this gets implemented ;)

[ioggstream]

@hairyhenderson

• surrounding with {{}}: should think about replacements conflicts, but probably it's fine.
• s/id/container_number/ :+1: explicit is better than implicit. If you sponsor this patch I'll make the changes asap.

nite + Peace,
R.

[StefanScherer]

@ioggstream Sorry, I had no time to dig into the environment section yet.

+1 for the suggestions from @hairyhenderson with {{ }} instead of %% %%, looks and feels more familiar with the other docker tools. Also container_number instead of id feels better!

Would love to see this to be merged soon!

[leriomaggio]

+1

[cgcgbcbc]

+1 for this feature

[aanand]

After some thought, I don't think this is a good idea.

1. We might well get rid of scale numbers in future. Once we move to multi-host systems with Swarm, keeping track of an incrementing sequence of integers will become (a) costly and (b) unreliable.
2. Even if we kept scale numbers, I'm not convinced it's a good idea to rely on them for dividing up work between containers. Better for the container to generate an identifier of its own and use that, so it's not dependent on Compose to work properly.

[rvion]

Thanks for the PR, but...
I agree with the comment above, so -1 for the feature.
Too much risk here.
when I see PR like that, I have the strong impression that docker-compose will end buggy and not maintanable

[aanand]

By the way, instead of interpolating the hostname in the command string (command: runme.sh %%hostname%%), you can just run the hostname command inside your shell script to get it.

[ioggstream]

2015-05-14 17:47 GMT+02:00 Aanand Prasad notifications@github.com:

instead of interpolating the hostname in the command string (command: runme.sh %%hostname%%), you can just run the hostname command inside your shell script to get it.

This has the issue of multiplying the images.

Eg. I need two different images with different entrypoints just to
change the arguments (while this is actually the goal of the "command"
section).

[dnephin]

@ioggstream you don't need separate images at all, you can do this:

CMD  runme.sh $(hostname)

or

command: runme.sh $(hostname)

[cgcgbcbc]

Eg. I need two different images with different entrypoints just to
change the arguments (while this is actually the goal of the "command"
section).

My use case is similar to yours. But after some thoughts I think it is bad design if the argument need to be specified in docket-compose.yml

[ioggstream]

Thanks @ALL for your replies!

2015-05-14 18:08 GMT+02:00 Daniel Nephin notifications@github.com:

@ioggstream you can do this:

CMD runme.sh $(hostname)
Wouldn't this lead to a new image?

command: runme.sh $(hostname)
iiuc about how docker-compose works, this line should not work as expected.

It causes

execve('runme.sh', 'runme.sh', '$(hostname)', ...)

While for this to work I should have bash powers, eg. something clumsy
like this, which requires entrypoint==/bin/sh

entrypoint: /bin/sh
command: -c 'runme.sh $(hostname)'

Let me know if I'm wrong...

Thx again + Peace,
R.

[ioggstream]

@cgcgbcbc

I think it is bad design if the argument need to be specified in docket-compose.yml

I think it depends :) Can you please share your POV about it?

[aanand]

@ioggstream You can also just run the hostname command inside runme.sh, rather than trying to pass it in as a command line argument.

Could you explain your use case? Why do you need your container's hostname to contain the scale number?

[cgcgbcbc]

I think it depends :) Can you please share your POV about it?

Yep, that's depends. For example, if the argument is something like command: start master or command: start slave, the argument is independent with the container and depend on what service are to run. But when it comes to my use case: command: start slave -h slave_{{number}} and hostname: slave_{{number}}, I think I should change my scripts so that it can determine the hostname itself on start.

[ioggstream]

@aanand @cgcgbcbc before all thanks to you and the others for your time ;)

Use case:

• running database and application server instances;
• databases may or may not be clustered (eg. I don't want --server-id or --slave-number in every command:)
• databases may be stopped and started, so IP is variable and links don't work;
• containers are discovered via DNS;

Using {{hostname}} and {{id}} in command I can:

• assign an unique id to every server;
• use the same entrypoint for various use case;
• so just one docker image on docker-hub running eg. mysqld.

[cgcgbcbc]

@ioggstream What is your mysql cluster design? But I think you can always rely on service discovery to make things work.

databases may be stopped and started, so IP is variable and links don't work;
containers are discovered via DNS;

You can use a mysql cluster proxy, and use the dns record for that proxy in your application server

[ioggstream]

2015-05-18 4:30 GMT+02:00 Guang Chen notifications@github.com:

@ioggstream https://github.com/ioggstream What is your mysql cluster
design?

I don't want the image to be tied to any prior design.

Instead I want them to be flexible like I was running them from command
line.

But I think you can always rely on service discovery to make things work.

—
Reply to this email directly or view it on GitHub
#1131 (comment).

[cgcgbcbc]

I don't want the image to be tied to any prior design.

I'm not familiar with database, but when you say mysql cluster, do you mean the mysql cluster with sql node, data node and management node or simply mysql master-slave replication ?

[cgcgbcbc]

--server-id

for --server-id I think this is an unscalable design for mysql, so I would prefer some tricks to generate the unique id by some other unique flags inside the container, for example, use the 32 bit ip-v4 address

[ioggstream]

2015-05-18 14:48 GMT+02:00 Guang Chen notifications@github.com:

I would prefer some tricks to generate the unique id

I don't want my entrypoint.sh to be tied to any configuration.

The image should let you free to use or not --server-id, --bin-log & co.

I want a general docker image with no specialization, otherwise I would
need, maintain and push to docker-hub more images.

[cgcgbcbc]

otherwise I would need, maintain and push to docker-hub more images

I don't quite understand why need to generate more images in this case, since arguments can be passed. Suppose there is a flag --gen-server-id, if passed, the entrypoint.sh will generate a server-id and use that, otherwise omit the --server-id parameter when starting mysqld. The only disadvantage but not really a disadvantage is that the entrypoint.sh will be more complex

[ioggstream]

2015-05-18 16:27 GMT+02:00 Guang Chen notifications@github.com:

Suppose there is a flag --gen-server-id

This approach is perfectly legitimate for specialized images. In the mysql
case you'll end in tens of --PARAMETERS.

And then you have to reimplement this for every application (eg wildfly,
...).

I just want the entrypoint.sh be a proxy to mysqld/startup.sh/whatever.

In whatever way you want to run mysql/wildfly/xxx (cluster, replica & co)
you can just set the arguments in the command and save your
docker-compose.yml

Note then, that parameterized hostnames mimics exactly what happens with
container names.

[cgcgbcbc]

case you'll end in tens of --PARAMETERS.

If this is the production case, I think a configuration delivery server is needed.

[jdmarshall]

I'm perfectly okay with compose generating the names, as long as the names don't change between restarts. What do we need to do to get that to happen?

[ioggstream]

2015-05-20 3:46 GMT+02:00 Jason Marshall notifications@github.com:

I'm perfectly okay with compose generating the names, as long as the names
don't change between restarts.

iiuc unless you don't remove the containers, names and hostnames should be
preserved.

If you remove them:

• names are deterministically generated, so are somewhat preserved
• hostnames are random
• ips are random too

What do we need to do to get that to happen?

I think you can't preserve hostnames between restart because random hostnames are probably unique in the sense of "temporal locality" (eg. docker ensures every hostname is unique in the moment of generation).

If you re-run compose after a rm, one of your random hostname could have
been taken from another container.

The only way is to pass hostnames to compose.

[dnephin]

To recap some of the discussion:

• $(hostname) or $(hostname -i) can be used to get a unique id, or the ip address of a scaled container. Using this does not duplicate images, it's evaluated at container run time.
• we'd like to remove the sequential scale ids and only use the uniqueness of the container ids. They are not reliable in a distributed system (see Proposal: Stop using sequential container numbers #1516).
• we'd like to avoid adding more variables. We already have variable interpolation in the compose file itself.

Since there is always a way to do this (using the unique hostname), it's unlikely we will be able to accept this PR.

[allingeek]

While $(hostname -i) does resolve to the IP address of a scaled container, $(hostname) does not resolve to the container name at all. As of 1.5 it resolves to the service name. Now, if I were building my own images or writing my own scripts I could code around this by just using some random ID or the container IP address. But I'm using off the shelf images that detect and use the hostname as reported by $(hostname). That is a problem for scaled containers because they are all "registering" the same name. Can I do a bunch of work to code around this? Sure. But honestly, this feels more like a concern for the scaling infrastructure (Compose) that is already managing these conflicts.

$ docker exec -ti stack_service_1 /bin/sh
/ # hostname
service
$ docker exec -ti stack_service_2 /bin/sh
/ # hostname
service

This might not be a problem if it weren't for that pesky UTS namespace. I could simply add a quick hostname $(< /dev/urandom tr -dc a-z-0-9 | head -c${1:-32};echo;) Changing the hostname from within a container would require CAP_SYS_ADMIN and granting that is WAY overkill.

[dnephin]

Setting the hostname to a service name will be fixed in 1.5.1 ( #2334)

[dnephin]

As discussed in #1131 (comment) we are not going to be able to merge this PR, so I'm going to close it.

There is another proposal in #1503 that I think may provide another option to solve this problem (in addition to the existing option of using $(hostname) or $(hostname -i).