minimesos fails to start on Fedora 23

[sadovnikov]

This bug was reported at http://www.meetup.com/Software-Circus/events/228889225/

Symptoms are:

• although ZooKeeper image is pre-downloaded by bash script, minimesos starts and fails to download it too;
• after that minimesos container is seen with created status - like it has not run yet, although logs are seen in console

[sadovnikov]

we'll remove from 0.7.1 milestone unless it's an easy fix

[sadovnikov]

Need some more info about running docker and minimesos on Linux. Both seem to require sudo

[sadovnikov]

Here is the problem: docker images on Fedora prefixes images with the repository.

[vagrant@localhost ~]$ docker images
REPOSITORY                            TAG                        IMAGE ID            CREATED             VIRTUAL SIZE
docker.io/containersol/minimesos      0.7.0                      c0c11fa997c0        4 days ago          130.5 MB
docker.io/containersol/mesos-agent    0.25.0-0.2.70.ubuntu1404   d8c3d041ce51        4 days ago          468.1 MB
docker.io/containersol/mesos-master   0.25.0-0.2.70.ubuntu1404   184ea6793be4        4 days ago          468.1 MB
docker.io/jplock/zookeeper            3.4.6                      20665487553b        2 weeks ago         163.2 MB
[vagrant@localhost ~]$ docker images containersol/minimesos
REPOSITORY          TAG                 IMAGE ID            CREATED             VIRTUAL SIZE

This makes pullImage() in bash script and, seems, the code to fail to detect existence of downloaded image

pullImage() {
    if [ "$(docker images $1 | grep $2 2> /dev/null)" = "" ]; then
    echo "Pulling $1:$2"
    docker pull "$1:$2"
    fi
}

Pulling jplock/zookeeper:3.4.6
Trying to pull repository docker.io/jplock/zookeeper ... 3.4.6: Pulling from jplock/zookeeper
Digest: sha256:122df5321afdf89480598d933d2f350d0cb9a95954f9eafb525307ac0d14a2c8
Status: Image is up to date for docker.io/jplock/zookeeper:3.4.6

[rio]

Yes, docker daemon always runs as root. To be able to use it users can be added to the docker group. Usually /var/run/docker.sock has docker as a group but it is essentially root

Tripped me up in the beginning too.

[sadovnikov]

Hi @rio, thanks for giving yourself away ;)
As turns out, Fedora comes with RedHat flavour of docker https://github.com/projectatomic/docker/tree/fedora-1.10, which is different.

One of the difference is registering downloaded images with repository they are downloaded from (you won't see docker.io/ in normal docker). Now I'm figuring out if we can come up bash and Java code, which would work in both versions of docker

[rio]

That would be great. I why they did it has something to do with supporting their openshift platform and multiple registry sources.

[jbirch]

This is (totally expectedly) also failing under Fedora 24 for the same reasons. It's an easy enough workaround for now to just create some additional tags. It runs into an issue with some NPEs at some point, but destroying and recreating the cluster allows it to pull the agent image, which can be tagged. Things work as expected then.

[jbirch@sleepygary ~/Workspace/scratch/minimesos]$ minimesos up
Pulling of jplock/zookeeper:3.4.6 completed. However the image is not found

[jbirch@sleepygary ~/Workspace/scratch/minimesos]$ docker tag docker.io/jplock/zookeeper:3.4.6 jplock/zookeeper:3.4.6

[jbirch@sleepygary ~/Workspace/scratch/minimesos]$ minimesos up
Pulling of containersol/mesos-master:0.25.0-0.2.70.ubuntu1404 completed. However the image is not found

[jbirch@sleepygary ~/Workspace/scratch/minimesos]$ docker tag docker.io/containersol/mesos-master:0.25.0-0.2.70.ubuntu1404 containersol/mesos-master:0.25.0-0.2.70.ubuntu1404

[jbirch@sleepygary ~/Workspace/scratch/minimesos]$ minimesos up
Cluster 1462473841 is already running

[jbirch@sleepygary ~/Workspace/scratch/minimesos]$ minimesos 
Minimesos cluster is running: 1462473841
Mesos version: 0.25
Exception in thread "main" java.lang.NullPointerException
    at com.containersol.minimesos.main.CommandInfo.printServiceUrls(CommandInfo.java:61)
    at com.containersol.minimesos.main.CommandInfo.execute(CommandInfo.java:44)
    at com.containersol.minimesos.main.Main.run(Main.java:108)
    at com.containersol.minimesos.main.Main.main(Main.java:56)

[jbirch@sleepygary ~/Workspace/scratch/minimesos]$ minimesos destroy
Destroyed minimesos cluster with ID 1462473841

[jbirch@sleepygary ~/Workspace/scratch/minimesos]$ minimesos up
Pulling of containersol/mesos-agent:0.25.0-0.2.70.ubuntu1404 completed. However the image is not found

[jbirch@sleepygary ~/Workspace/scratch/minimesos]$ docker tag docker.io/containersol/mesos-agent:0.25.0-0.2.70.ubuntu1404 containersol/mesos-agent:0.25.0-0.2.70.ubuntu1404

[jbirch@sleepygary ~/Workspace/scratch/minimesos]$ minimesos up
Cluster 2818956538 is already running

[jbirch@sleepygary ~/Workspace/scratch/minimesos]$ minimesos
Minimesos cluster is running: 2818956538
Mesos version: 0.25
export MINIMESOS_NETWORK_GATEWAY=172.17.0.1
export MINIMESOS_ZOOKEEPER=zk://172.17.0.3:2181/mesos; export MINIMESOS_ZOOKEEPER_IP=172.17.0.3
export MINIMESOS_MASTER=http://172.17.0.4:5050; export MINIMESOS_MASTER_IP=172.17.0.4

After that, things appear alright. I've yet to hit it in anger, but this seems to be a relatively painless workaround when manually dealing with minimesos.

edit: slight lie, you've gotta keep destroying and upping to find that you're missing things like marathon, etc. But if you just want mesos, this seems sufficient.