Docker image error: LOG: invalid length of startup packet

[khalid-hussain]

Prerequisites

• Put an X between the brackets on this line if you have done all of the following:
  • Checked the online documentation: https://mimic.physionet.org/about/mimic/
  • Checked that your issue isn't already addressed: https://github.com/MIT-LCP/mimic-code/issues?utf8=%E2%9C%93&q=

Description

Greetings,

I'm having a little trouble setting up the docker image. I've followed the guide here: https://github.com/MIT-LCP/mimic-code/tree/master/buildmimic/docker

My System:

------------------
System Information
------------------
Operating System: Windows 10 Pro 64-bit (10.0, Build 18363) (18362.19h1_release.190318-1202)
    Language: English (Regional Setting: English)
Processor: Intel(R) Core(TM) i7-3520M CPU @ 2.90GHz (4 CPUs), ~2.9GHz
Memory: 16384MB RAM
Available OS Memory: 16080MB RAM
Page File: 14726MB used, 10058MB available
Windows Dir: C:\WINDOWS

My current folder tree at (D:):

.
|-- mimic_code
|   |-- benchmark
|   |-- buildmimic
|   |-- concepts
|   |-- notebooks
|   |-- tests
|   `-- tutorials
|-- mimic_data
|   `-- csv
`-- mimic_pgdata

The folder at \mimic-data\csv contains the MIMIC-III .csv.gz data files. The folder mimic_pgdata is empty at the time of running the container. It's empty even after running the container. I assume this is because I do not execute any commands after the container is created.

The command I used (using the Git Bash terminal):

docker run \
--name mimic \
-p 5433:5432 \
-e BUILD_MIMIC=1 \
-e POSTGRES_PASSWORD=postgres \
-e MIMIC_PASSWORD=mimic \
-v //d/mimic_data/csv:/mimic_data \
-v //d/mimic_pgdata:/var/lib/postgresql/data \
-d postgres/mimic

My result from the Docker Dashboard:

PostgreSQL Database directory appears to contain a database; Skipping initialization
2020-02-04 15:23:25.966 UTC [1] LOG:  starting PostgreSQL 12.1 (Debian 12.1-1.pgdg100+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
2020-02-04 15:23:25.974 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
2020-02-04 15:23:25.974 UTC [1] LOG:  listening on IPv6 address "::", port 5432
2020-02-04 15:23:25.995 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2020-02-04 15:23:26.014 UTC [25] LOG:  database system was shut down at 2020-02-04 15:20:26 UTC
2020-02-04 15:23:26.020 UTC [1] LOG:  database system is ready to accept connections
2020-02-04 15:23:26.034 UTC [32] LOG:  invalid length of startup packet
2020-02-04 15:23:26.086 UTC [33] LOG:  invalid length of startup packet
2020-02-04 15:23:27.219 UTC [34] LOG:  invalid length of startup packet
2020-02-04 15:23:27.219 UTC [35] LOG:  invalid length of startup packet
2020-02-04 15:23:28.257 UTC [36] LOG:  invalid length of startup packet
2020-02-04 15:23:28.259 UTC [37] LOG:  invalid length of startup packet
2020-02-04 15:23:29.264 UTC [38] LOG:  invalid length of startup packet
2020-02-04 15:23:29.269 UTC [39] LOG:  invalid length of startup packet
2020-02-04 15:23:29.795 UTC [40] LOG:  invalid length of startup packet
2020-02-04 15:23:30.278 UTC [42] LOG:  invalid length of startup packet
2020-02-04 15:23:30.278 UTC [41] LOG:  invalid length of startup packet
2020-02-04 15:23:30.824 UTC [43] LOG:  invalid length of startup packet
2020-02-04 15:23:31.287 UTC [44] LOG:  invalid length of startup packet
2020-02-04 15:23:31.292 UTC [45] LOG:  invalid length of startup packet
2020-02-04 15:23:31.834 UTC [46] LOG:  invalid length of startup packet
2020-02-04 15:23:32.854 UTC [47] LOG:  invalid length of startup packet
2020-02-04 15:23:33.863 UTC [48] LOG:  invalid length of startup packet
2020-02-04 15:23:34.875 UTC [49] LOG:  invalid length of startup packet

The output of docker images:

REPOSITORY                           TAG                 IMAGE ID            CREATED             SIZE
postgres/mimic                       latest              10d900d346ac        19 minutes ago      440MB
postgres                             latest              cf879a45faaa        2 days ago          394MB
k8s.gcr.io/kube-apiserver            v1.15.5             e534b1952a0d        3 months ago        207MB
k8s.gcr.io/kube-controller-manager   v1.15.5             1399a72fa1a9        3 months ago        159MB
k8s.gcr.io/kube-proxy                v1.15.5             cbd7f21fec99        3 months ago        82.4MB
k8s.gcr.io/kube-scheduler            v1.15.5             fab2dded59dd        3 months ago        81.1MB
golang                               1.11-alpine         e116d2efa2ab        5 months ago        312MB
docker/kube-compose-controller       v0.4.23             a8c3d87a58e7        8 months ago        35.3MB
docker/kube-compose-api-server       v0.4.23             f3591b2cb223        8 months ago        49.9MB
alpine                               3.5                 f80194ae2e0c        12 months ago       4MB
k8s.gcr.io/coredns                   1.3.1               eb516548c180        12 months ago       40.3MB
k8s.gcr.io/etcd                      3.3.10              2c4adeb21b4f        14 months ago       258MB
k8s.gcr.io/pause                     3.1                 da86e6ba6ca1        2 years ago         742kB

I've tried:

1. deleting the image and recreating it again but to no avail.
2. different HOST port numbers but I still get the same output.
3. using a single slash for addresses in running the container, but it failed despite using the Git Bash terminal.

[alistairewj]

Honestly not that familiar with Docker - but maybe @xiaosongz or @haobibo could comment since they both have recently been working with the build.

[haobibo]

Hi @khalid-hussain , @xiaosongz has found a bug in the docker image build file setup.sh.
And I've created a PR #693 trying to fix that.

Could you please follow the updated setup.sh (or directly clone it from https://github.com/haobibo/mimic-code) to see if the updated code works or not.

[khalid-hussain]

I'm sorry, I got preoccupied with some other issues. I'll give it a try tomorrow. Thanks for the prompt replies.

[khalid-hussain]

UPDATE: I'm still having the same problem. I updated the setup.sh and the Dockerfile as well but it still gives me the same ouput.

[caiqing0204]

Hello, I have also found this problem. Is there any solution now?

[gimpiron]

+1

[Yuqi92]

The same issue here.

[Jonas-360ty]

+1

[Jonas-360ty]

Did anybody has a solution! Same problem with our system.

[bipin-num]

+1

[teohrt]

+1

[alistairewj]

+1

[nateloker]

+1

[Zonalds]

+1

[bringmetheaugust]

+1

[SerbanG]

+1

[HamedJP]

any solution?

[PedroMontesinos]

God please help, i've been stuck here for hours

[tompollard]

Someone in the lab may take a look at fixing the the Docker image at some point, but we have limited resources and this is not one of our priorities. We would be happy to review a pull request from the community.

[ghost]

Same issue here

[AhHa45]

same issue here

[yogeshk04]

I'm facing same issue?

-03-11 08:31:31.885 UTC [1] LOG: starting PostgreSQL 13.2 (Debian 13.2-1.pgdg100+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
2021-03-11 08:31:31.885 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
2021-03-11 08:31:31.885 UTC [1] LOG: listening on IPv6 address "::", port 5432
2021-03-11 08:31:31.899 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2021-03-11 08:31:31.914 UTC [66] LOG: database system was shut down at 2021-03-11 08:31:31 UTC
2021-03-11 08:31:31.924 UTC [1] LOG: database system is ready to accept connections
2021-03-11 08:31:51.485 UTC [73] FATAL: password authentication failed for user "pgadmin"
2021-03-11 08:31:51.485 UTC [73] DETAIL: Role "pgadmin" does not exist.
Connection matched pg_hba.conf line 99: "host all all all md5"
2021-03-11 08:32:00.969 UTC [74] FATAL: password authentication failed for user "pgadmin"
2021-03-11 08:32:00.969 UTC [74] DETAIL: Role "pgadmin" does not exist.
Connection matched pg_hba.conf line 99: "host all all all md5"
2021-03-11 08:32:06.826 UTC [75] FATAL: password authentication failed for user "pgadmin"
2021-03-11 08:32:06.826 UTC [75] DETAIL: Role "pgadmin" does not exist.
Connection matched pg_hba.conf line 99: "host all all all md5"
2021-03-11 08:33:38.128 UTC [78] FATAL: password authentication failed for user "pgadmin"
2021-03-11 08:33:38.128 UTC [78] DETAIL: Role "pgadmin" does not exist.
Connection matched pg_hba.conf line 99: "host all all all md5"
2021-03-11 08:33:52.780 UTC [79] LOG: invalid length of startup packet
2021-03-11 08:33:52.783 UTC [80] LOG: invalid length of startup packet..........................................

[Qichenye]

Same issue

[nololie]

+1

[MariaIsabel68]

Same

[WeslleyFratini]

Same here

[hyeseong-dev]

Same here

[droosifer]

same

[cleversonferreira]

same

[arrudacaio]

Same here

[tangjienan]

+1

[mfrocha-fiec]

+1

[matheus-de-sa]

+1

[dancernogorsky]

+1

[YSHIDM]

+1

march in step

[SkourtsidisGiorgos]

For me, it seems that the root cause is Prometheus, as hinted here: https://www.alibabacloud.com/blog/how-postgresql-12-generates-less-log-data-when-monitoring-and-detecting-incomplete-startup-packet_597767

Anybody found a fix/workaround?

[ann-aot]

same here. any fix?

[johndpope]

Not a complete solution - but if you're on mac -
make sure you have psql installed.
https://stackoverflow.com/questions/44654216/correct-way-to-install-psql-without-full-postgres-on-macos
confirm connection
psql -h 0.0.0.0 -p 5432 -U postgres (actually logs in now / though my mysqlworkbench isn't working)

[henning410]

2 years and no solution yet?

[tompollard]

@henning410 if you'd like to try to outline what you think are the necessary steps to fix this, then I can take a look. I'm not familiar with the Docker implementation and it looks like it was contributed by a member of the research community.

We're a very small research group at MIT building and maintaining both PhysioNet and MIMIC, so we really have to rely on the community for help with things like this.

[tompollard]

Side note, but I would think in many or most cases the problem may be solved by instead using the copies of MIMIC available to you in the cloud.

For an example notebook that references a PhysioNet database on Google BigQuery, see:
https://github.com/MIT-LCP/sccm-datathon/blob/master/01_explore_patients.ipynb

You can open this notebook directly in Google Colab with the link below:
https://colab.research.google.com/github/MIT-LCP/sccm-datathon/blob/master/01_explore_patients.ipynb

You'd just need to replace project_id='sccm-datathon' with your personal project ID and swap the query that references eicu_crd_demo with a query that references the relevant MIMIC dataset.

See the following discussion for some pointers on how to set up the cloud access with your PhysioNet account:
#1154

[tompollard]

@henning410 just dropped you an email to see if we can get this resolved. If anyone else is interested in helping out, please add a message here and I'll reach out.

[kossme]

for me solved by reducing the name length of the sql file that I was providing to /docker-entrypoint-initdb.d
in volumes.

[Pranav-Shinde]

I updated the docker and restarted it after clicking on 'Quit Docker Desktop' on top docker icon.
Its working now.

[divyankm]

+1

[polemeni]

+1

[elDaniello]

+1

[runatyr1]

For me this was solved by adding this to .helmignore:
templates/tests/
Also disabled the liveliness and readiness probes for the deployment (kuberentes).
Not sure if both are needed but I don't want to start making a lot of tests to figure that out at this moment, happy enough that it works :)
It looks like either or both the helm test pod or the deployment probes were sending malformed packets to the db and it got scared and shut down.

[hxshandle]

still open for years?
Any solutions?

[tompollard]

The Docker build script for MIMIC-III was contributed by a member of the research community a few years back.

The script is now stale, and this is clearly a source of of frustration. We don't have time to fix it ourselves unfortunately, so my vote would be to remove the Docker folder from this repo. @alistairewj, good with you?

If someone would like to have a go at adding build scripts later, we'd be happy to review and merge a pull request.

[alistairewj]

yep, seems fair, PR in #1635

[tompollard]

Closed with #1635. The broken Docker build scripts have been removed.

[RapalS]

i have docker compose script working properly for mimiciv and mimiciv-ed. will upload once get chance. DM me if you want the scripts as it is now.