cloudtunes-worker and cloudtunes-8000 exiting status 1 when using docker

[lukkamor]

Successfully built 505b6bb4e9e1
root@test:~/cloudtunes-master# docker images
REPOSITORY             TAG                 IMAGE ID            CREATED             VIRTUAL SIZE
cloudtunes-img         latest              505b6bb4e9e1        12 seconds ago      946.7 MB
dockerfile/ubuntu      latest              29775a775590        10 hours ago        416.2 MB
tleyden5iwx/open-ocr   latest              a9fa81953639        5 days ago          1.566 GB
tutum/rabbitmq         latest              4c56c7f92bf7        7 days ago          264.2 MB
ubuntu                 latest              826544226fdc        7 days ago          194.2 MB
root@test:~/cloudtunes-master# docker run --name=cloudtunes --publish=8000:8000  --detach --tty cloudtunes-img
96d8da51aabff9de3800755a884e322c31e1c6980007c02f3886f62f42d72276
root@test:~/cloudtunes-master# docker logs 96d8
/usr/lib/python2.7/dist-packages/supervisor/options.py:295: UserWarning: Supervisord is running as root and it is searching for its configuration file in default locations (including its current working directory); you probably want to specify a "-c" argument specifying an absolute path to a configuration file for improved security.
  'Supervisord is running as root and it is searching '
2014-09-12 17:12:42,533 CRIT Supervisor running as root (no user in config file)
2014-09-12 17:12:42,533 WARN Included extra file "/etc/supervisor/conf.d/cloudtunes.conf" during parsing
2014-09-12 17:12:42,567 INFO RPC interface 'supervisor' initialized
2014-09-12 17:12:42,567 CRIT Server 'unix_http_server' running without any HTTP authentication checking
2014-09-12 17:12:42,567 INFO supervisord started with pid 1
2014-09-12 17:12:43,575 INFO spawned: 'cloudtunes-worker' with pid 8
2014-09-12 17:12:43,582 INFO spawned: 'redis' with pid 9
2014-09-12 17:12:43,589 INFO spawned: 'mongod' with pid 10
2014-09-12 17:12:43,596 INFO spawned: 'cloudtunes-8000' with pid 11
2014-09-12 17:12:43,805 INFO exited: cloudtunes-worker (exit status 1; not expected)
2014-09-12 17:12:43,805 INFO exited: cloudtunes-8000 (exit status 1; not expected)
2014-09-12 17:12:45,070 INFO spawned: 'cloudtunes-worker' with pid 28
2014-09-12 17:12:45,071 INFO success: redis entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2014-09-12 17:12:45,071 INFO success: mongod entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2014-09-12 17:12:45,077 INFO spawned: 'cloudtunes-8000' with pid 29
2014-09-12 17:12:45,258 INFO exited: cloudtunes-worker (exit status 1; not expected)
2014-09-12 17:12:45,259 INFO exited: cloudtunes-8000 (exit status 1; not expected)
2014-09-12 17:12:47,267 INFO spawned: 'cloudtunes-worker' with pid 34
2014-09-12 17:12:47,272 INFO spawned: 'cloudtunes-8000' with pid 35
2014-09-12 17:12:47,442 INFO exited: cloudtunes-worker (exit status 1; not expected)
2014-09-12 17:12:47,445 INFO exited: cloudtunes-8000 (exit status 1; not expected)
root@test:~/cloudtunes-master# docker ps
CONTAINER ID        IMAGE                   COMMAND                CREATED             STATUS              PORTS                    NAMES
96d8da51aabf        cloudtunes-img:latest   supervisord --nodaem   56 seconds ago      Up 55 seconds       0.0.0.0:8000->8000/tcp   cloudtunes

[jkbrzt]

I see. It's probably because supervisor starts all the services at the same time, but CloudTunes needs a Mongod DB and a Redis connections upon boot. I will look into this over the weekend.

[lukkamor]

Thanks so much Jakub!
Arturo

On Friday, September 12, 2014 1:51 PM, Jakub Roztočil notifications@github.com wrote:

I see. It's probably because supervisors starts all the services at the same time, but CloudTunes needs Mongod DB and Redis connections upon boot. I will look into this over the weekend.
—
Reply to this email directly or view it on GitHub.

[petems]

Hi! I figured this out (I ended up making a bash container and looking at the error logs http://affy.blogspot.co.uk/2014/06/how-to-detach-from-running-docker-image.html 👍)

So, if you dont have a local.py file, you wont be able to start cloudtunes-server:

!!!Cannot import local settings!!!

You need to copy cloudtunes/settings/local.example.py
to cloudtunes/settings/local.py and fill in the None's.

https://github.com/jakubroztocil/cloudtunes#installation

So, I just added that file to the repo and the dockerfile worked fine.

I'm no docker expert, but I assume something like this would work:

ADD cloudtunes-server/cloudtunes/settings/local.example.py /home/cloudtunes-server/cloudtunes/settings/local.py

[petems]

Actually, we should probably add instructions to the README.md to create the local.py file in that directory? As you'll need to configure your own personal keys in there...

[petems]

PR here: #20

[lukkamor]

Thanks Petems, I have added the local.py and still have the same problem. I think Jakub is looking at the way the components start and that might fix the problem.

[kakawait]

@lukkamor You should patch supervisor.ini like following

From aeb50b20ff997055cf0b1cff56e9fc8294d55b4c Mon Sep 17 00:00:00 2001
From: kakawait <thibaud.lepretre@gmail.com>
Date: Mon, 22 Sep 2014 18:23:42 +0200
Subject: [PATCH] fix supervisor starting order

---
 cloudtunes-server/production/supervisor.ini | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/cloudtunes-server/production/supervisor.ini b/cloudtunes-server/production/supervisor.ini
index aeafd09..b950ee6 100644
--- a/cloudtunes-server/production/supervisor.ini
+++ b/cloudtunes-server/production/supervisor.ini
@@ -11,6 +11,7 @@ autorestart=true
 redirect_stderr=true
 stdout_logfile=/var/log/cloudtunes-server.log
 loglevel=debug
+priority=998


 ;[program:cloudtunes-8001]
@@ -33,6 +34,7 @@ autorestart=true
 redirect_stderr=true
 stdout_logfile=/var/log/cloudtunes-worker.log
 loglevel=debug
+priority=999

 [program:redis]
 directory=/data
@@ -42,9 +44,11 @@ autorestart=true
 user=root
 stdout_logfile=/var/log/redis/stdout.log
 stderr_logfile=/var/log/redis/stderr.log
+priority=2

 [program:mongod]
 command=/usr/bin/mongod --smallfiles
 stdout_logfile=/var/log/supervisor/%(program_name)s.log
 stderr_logfile=/var/log/supervisor/%(program_name)s.log
 autorestart=true
+priority=1
-- 
1.8.4.2

It works for me.

upstream problem is following: Supervisor/supervisor#122 but still open since 2012 :'(

[lukkamor]

Hi Jakub,
thanks for the info. Still the same problem. I've just downloaded cloudtunes-master.zip, applied the patch to supervisor.ini, add the local.py file and still have the same problem. Maybe I am doing something wrong. I have the log file with all the building steps. Would that help? I am stuck.
Thanks!

On Monday, September 22, 2014 12:26 PM, Thibaud Lepretre notifications@github.com wrote:

@lukkamor You should patch supervisor.ini like following
From aeb50b20ff997055cf0b1cff56e9fc8294d55b4c Mon Sep 17 00:00:00 2001
From: kakawait thibaud.lepretre@gmail.com
Date: Mon, 22 Sep 2014 18:23:42 +0200
Subject: [PATCH] test --- cloudtunes-server/production/supervisor.ini | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/cloudtunes-server/production/supervisor.ini b/cloudtunes-server/production/supervisor.ini
index aeafd09..b950ee6 100644
--- a/cloudtunes-server/production/supervisor.ini
+++ b/cloudtunes-server/production/supervisor.ini
@@ -11,6 +11,7 @@ autorestart=true redirect_stderr=true stdout_logfile=/var/log/cloudtunes-server.log loglevel=debug
+priority=998 ;[program:cloudtunes-8001]
@@ -33,6 +34,7 @@ autorestart=true redirect_stderr=true stdout_logfile=/var/log/cloudtunes-worker.log loglevel=debug
+priority=999 [program:redis] directory=/data
@@ -42,9 +44,11 @@ autorestart=true user=root stdout_logfile=/var/log/redis/stdout.log stderr_logfile=/var/log/redis/stderr.log
+priority=2 [program:mongod] command=/usr/bin/mongod --smallfiles stdout_logfile=/var/log/supervisor/%(program_name)s.log stderr_logfile=/var/log/supervisor/%(program_name)s.log autorestart=true

+priority=1

1.8.4.2
It works for me.
upstream problem is following: Supervisor/supervisor#122 but still open since 2012 :'(
—
Reply to this email directly or view it on GitHub.

[kakawait]

@lukkamor Yeah my bad my patch works "sometimes"... The only solution I found is to add the following directive inside [program:cloudtunes-8000] and [program:cloudtunes-worker]

startretries=10
startsecs=10

The goal is to increase the number of retries before giving up. With 10 retries on my docker env it work, mongodb and redis had enough time to start. But you can tune it.

[lukkamor]

Tried: 20 retries and 30 seconds unsuccessfully. One thing I've noticed when building, I got all this errors:
Step 12 : RUN gem install compass
---> Running in 4439341c025e
Building native extensions. This could take a while...
ESC[91mEnclosing class/module 'moduleFFI' for class FunctionType not known
ESC[0mESC[91mEnclosing class/module 'moduleFFI' for class ArrayType not known
ESC[0mESC[91mEnclosing class/module 'moduleFFI' for class Buffer not known
ESC[0mESC[91mEnclosing class/module "BufferClass" for alias length total not known
ESC[0mESC[91mEnclosing class/module 'rbffi_TypeClass' for class Mapped not known
ESC[0mESC[91mEnclosing class/module 'moduleFFI' for module Platform not known
ESC[0mESC[91mEnclosing class/module 'moduleFFI' for class VariadicInvoker not known
ESC[0mESC[91mEnclosing class/module 'moduleFFI' for class Function not known
ESC[0mESC[91mEnclosing class/module 'moduleFFI' for class Struct not known
ESC[0mESC[91mEnclosing class/module 'rbffi_StructClass' for class InlineArray not known
Enclosing class/module 'rbffi_StructLayoutClass' for class CharArray not known
ESC[0mESC[91mEnclosing class/module "rbffi_StructLayoutCharArrayClass" for alias to_str to_s not known
ESC[0mESC[91mEnclosing class/module 'moduleFFI' for module DataConverter not known
ESC[0mESC[91mEnclosing class/module 'moduleFFI' for class StructByValue not known
ESC[0mESC[91mEnclosing class/module 'moduleFFI' for class StructLayout not known
ESC[0mESC[91mEnclosing class/module 'rbffi_StructLayoutClass' for class Field not known
ESC[0mESC[91mEnclosing class/module 'rbffi_StructLayoutClass' for class Number not known
ESC[0mESC[91mEnclosing class/module 'rbffi_StructLayoutClass' for class String not known
ESC[0mESC[91mEnclosing class/module 'rbffi_StructLayoutClass' for class Pointer not known
ESC[0mESC[91mEnclosing class/module 'rbffi_StructLayoutClass' for class Function not known
ESC[0mESC[91mEnclosing class/module 'rbffi_StructLayoutClass' for class Array not known
ESC[0mESC[91mEnclosing class/module 'moduleFFI' for class MemoryPointer not known
ESC[0mESC[91mEnclosing class/module 'moduleFFI' for module NativeType not known

are those normal?
Thanks!

On Tuesday, September 23, 2014 12:28 PM, Thibaud Lepretre notifications@github.com wrote:

Yeah my bad my patch works "sometimes"... The only solution I found is to add the following directive inside [program:cloudtunes-8000] and [program:cloudtunes-worker]
startretries=10
startsecs=10
The goal is to increase the number of retries before giving up. With 10 retries on my docker env it work but you can tune it.
—
Reply to this email directly or view it on GitHub.

[Lich4r]

I got the same problem