wire-server installation failed #470

JonnyX11 · 2021-06-13T10:20:05Z

Hello,

I got this error, Anyone know what to do ?

bash-4.4# helm upgrade --install wire-server wire/wire-server -f my-wire-server/values.yaml -f my-wire-server/secrets.yaml --wait

Error: UPGRADE FAILED: pre-upgrade hooks failed: timed out waiting for the condition

Can you please help?

arthurwolf · 2021-06-13T20:48:49Z

If you add --debug to the command, you might get some more information on what's going on.

…

On Sun, Jun 13, 2021 at 12:20 PM JonnyX11 ***@***.***> wrote: Hello, I got this error, Anyone know what to do ? bash-4.4# helm upgrade --install wire-server wire/wire-server -f my-wire-server/values.yaml -f my-wire-server/secrets.yaml --wait Error: UPGRADE FAILED: pre-upgrade hooks failed: timed out waiting for the condition Can you please help? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#470>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAA2SFN5QQY6TWCH5IG5JXTTSSA6DANCNFSM46TTVQJA> .

-- 勇気とユーモア

JonnyX11 · 2021-06-14T09:54:20Z

bash-4.4# helm upgrade --install wire-server wire/wire-server -f my-wire-server/values.yaml -f my-wire-server/secrets.yaml --wait --debug
history.go:56: [debug] getting history for release wire-server
upgrade.go:123: [debug] preparing upgrade for wire-server
upgrade.go:131: [debug] performing update for wire-server
upgrade.go:303: [debug] creating upgraded release for wire-server
client.go:282: [debug] Starting delete for "cassandra-migrations" Job
client.go:122: [debug] creating 1 resource(s)
client.go:491: [debug] Watching for changes to Job cassandra-migrations with timeout of 5m0s
client.go:519: [debug] Add/Modify event for cassandra-migrations: ADDED
client.go:558: [debug] cassandra-migrations: Jobs active: 0, jobs failed: 0, jobs succeeded: 0
client.go:519: [debug] Add/Modify event for cassandra-migrations: MODIFIED
client.go:558: [debug] cassandra-migrations: Jobs active: 1, jobs failed: 0, jobs succeeded: 0

upgrade.go:369: [debug] warning: Upgrade "wire-server" failed: pre-upgrade hooks failed: timed out waiting for the condition
Error: UPGRADE FAILED: pre-upgrade hooks failed: timed out waiting for the condition
helm.go:81: [debug] pre-upgrade hooks failed: timed out waiting for the condition
UPGRADE FAILED
main.newUpgradeCmd.func2
helm.sh/helm/v3/cmd/helm/upgrade.go:157
github.com/spf13/cobra.(*Command).execute
github.com/spf13/cobra@v1.1.1/command.go:850
github.com/spf13/cobra.(*Command).ExecuteC
github.com/spf13/cobra@v1.1.1/command.go:958
github.com/spf13/cobra.(*Command).Execute
github.com/spf13/cobra@v1.1.1/command.go:895
main.main
helm.sh/helm/v3/cmd/helm/helm.go:80
runtime.main
runtime/proc.go:225
runtime.goexit
runtime/asm_amd64.s:1371

JonnyX11 · 2021-06-15T11:44:43Z

bash-4.4# helm upgrade --install wire-server wire/wire-server -f my-wire-server/values.yaml -f my-wire-server/secrets.yaml --wait --debug
history.go:56: [debug] getting history for release wire-server
upgrade.go:123: [debug] preparing upgrade for wire-server
upgrade.go:131: [debug] performing update for wire-server
upgrade.go:303: [debug] creating upgraded release for wire-server
client.go:282: [debug] Starting delete for "cassandra-migrations" Job
client.go:122: [debug] creating 1 resource(s)
client.go:491: [debug] Watching for changes to Job cassandra-migrations with timeout of 5m0s
client.go:519: [debug] Add/Modify event for cassandra-migrations: ADDED
client.go:558: [debug] cassandra-migrations: Jobs active: 0, jobs failed: 0, jobs succeeded: 0
client.go:519: [debug] Add/Modify event for cassandra-migrations: MODIFIED
client.go:558: [debug] cassandra-migrations: Jobs active: 1, jobs failed: 0, jobs succeeded: 0
client.go:519: [debug] Add/Modify event for cassandra-migrations: MODIFIED
client.go:282: [debug] Starting delete for "elasticsearch-index-create" Job
client.go:122: [debug] creating 1 resource(s)
client.go:491: [debug] Watching for changes to Job elasticsearch-index-create with timeout of 5m0s
client.go:519: [debug] Add/Modify event for elasticsearch-index-create: ADDED
client.go:558: [debug] elasticsearch-index-create: Jobs active: 0, jobs failed: 0, jobs succeeded: 0
client.go:519: [debug] Add/Modify event for elasticsearch-index-create: MODIFIED
client.go:558: [debug] elasticsearch-index-create: Jobs active: 1, jobs failed: 0, jobs succeeded: 0
upgrade.go:369: [debug] warning: Upgrade "wire-server" failed: pre-upgrade hooks failed: timed out waiting for the condition
Error: UPGRADE FAILED: pre-upgrade hooks failed: timed out waiting for the condition
helm.go:81: [debug] pre-upgrade hooks failed: timed out waiting for the condition
UPGRADE FAILED
main.newUpgradeCmd.func2
helm.sh/helm/v3/cmd/helm/upgrade.go:157
github.com/spf13/cobra.(*Command).execute
github.com/spf13/cobra@v1.1.1/command.go:850
github.com/spf13/cobra.(*Command).ExecuteC
github.com/spf13/cobra@v1.1.1/command.go:958
github.com/spf13/cobra.(*Command).Execute
github.com/spf13/cobra@v1.1.1/command.go:895
main.main
helm.sh/helm/v3/cmd/helm/helm.go:80
runtime.main
runtime/proc.go:225
runtime.goexit
runtime/asm_amd64.s:1371

JonnyX11 · 2021-06-16T06:52:00Z

Anyone :(

arthurwolf · 2021-06-17T21:10:09Z

Are you installing the demo stuff, or the production stuff? Which pages in the documentation are you following exactly? Can you give us a complete log of the commands you type, and their output, from the beginning?

I don't have an idea right now of what might be going wrong, but providing this information might help somebody else find a solution for you easier than without that information.

Sorry for taking so long to answer, I completely missed the notification for your message in my inbox.

arthurwolf · 2021-06-17T21:11:35Z

How much RAM and how many CPUs do you have on the server? Can you give us the results of:

cat /proc/cpuinfo | grep processor | wc -l

free -m

As well as the result of «uptime» before and after the helm upgrade command ?

Thanks.

lucendio · 2021-06-18T09:21:17Z

Hello @JonnyX11

we are always happy to help, but we kindly ask people who reach out for help to be cooperative. So, would you mind following the template structure that we put in place for these kind of scenarios? You might notice some significant overlap between the questions lisyted in the template and the ones asked by @arthurwolf

Additionally to what is in the template, please provide us with some more context. For example, is it your first installation or are you trying to upgrade an installation? If the latter, did you check the release notes for the new version you installed - also for wire-server? Please note, you will find the code for each chart in there.

Lastly, please provide us with the output of kubectl get pods -o wide --all-namespaces.

Thank you.

JonnyX11 · 2021-06-20T07:07:08Z

first of all thank you very much for your help!

Basic information

On-premises: Yes
Cloud-Provider: nope
Installation type: production
Kubernetes version:
bash-4.4# kubectl version
Client Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.7", GitCommit:"1dd5338295409edcfff11505e7bb246f0d325d15", GitTreeState:"clean", BuildDate:"2021-01-13T13:23:52Z", GoVersion:"go1.15.5", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.7", GitCommit:"1dd5338295409edcfff11505e7bb246f0d325d15", GitTreeState:"clean", BuildDate:"2021-01-13T13:15:20Z", GoVersion:"go1.15.5", Compiler:"gc", Platform:"linux/amd64"}
Helm version:
bash-4.4# helm version
version.BuildInfo{Version:"v3.5.2", GitCommit:"", GitTreeState:"", GoVersion:"go1.16"}
Installed with Kubespray:
(Helm) Charts version:
bash-4.4# helm show chart wire/wire-server | grep -m 1 version
version: 2.108.0
List of installed top-level charts:
Other related technologies + version:

bash-4.4# ansible --version
ansible 2.9.12
config file = /wire-server-deploy/ansible/ansible.cfg
configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /nix/store/407c5v2x920kri79w0sb9gb1g82q28mg-python3.8-ansible-2.9.12/lib/python3.8/site-packages/ansible
executable location = /nix/store/407c5v2x920kri79w0sb9gb1g82q28mg-python3.8-ansible-2.9.12/bin/ansible
python version = 3.8.8 (default, Feb 19 2021, 11:04:50) [GCC 10.2.0]

How did you install Wire?

according to the productions docs:
https://docs.wire.com/how-to/install/version-requirements.html#persistence
https://docs.wire.com/how-to/install/helm-prod.html#helm-prod

I used the docker option for helm
https://docs.wire.com/how-to/install/dependencies.html#dependencies

I worked on Ubuntu 18.04 for all virtual machines

Name	Amount	CPU Cores	Memory (GB)	Disk Space (GB)
Cassandra	3	2	4	80
MinIO	3	1	2	400
ElasticSearch	3	1	2	60
Kubernetes³	3	6¹	8	40
Restund⁴	2	1	2	10
Per-Server Totals	—	11 CPU Cores	18 GB Memory	590 GB Disk Space
Admin Host²	1	1	4	40
Asset Host²	1	1	4	100
Per-Server Totals with Admin and Asset Hosts	—	13 CPU Cores	26 GB Memory	730 GB Disk Space

Please see the outputs here :

bash-4.4# kubectl version
Client Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.7", GitCommit:"1dd5338295409edcfff11505e7bb246f0d325d15", GitTreeState:"clean", BuildDate:"2021-01-13T13:23:52Z", GoVersion:"go1.15.5", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.7", GitCommit:"1dd5338295409edcfff11505e7bb246f0d325d15", GitTreeState:"clean", BuildDate:"2021-01-13T13:15:20Z", GoVersion:"go1.15.5", Compiler:"gc", Platform:"linux/amd64"}

bash-4.4# kubectl get pods -o NAMESPACE NAME default cassandra-migrations-d5g22 default demo-smtp-7f7cc8b585-bcrkq default elasticsearch-index-create-vm6zs default fake-aws-s3-77d9447b8f-2vzgj default fake-aws-s3-reaper-78d9f58dd4-zmqsb default fake-aws-sns-6c7c4b7479-c4thd default fake-aws-sqs-59fbfbcbd4-f7d72 default redis-ephemeral-master-0 kube-system coredns-7677f9bb54-4kg2d kube-system coredns-7677f9bb54-cnxm2 kube-system dns-autoscaler-5b7b5c9b6f-vxwcw kube-system kube-apiserver-kubenode01 kube-system kube-apiserver-kubenode02 kube-system kube-apiserver-kubenode03 kube-system kube-controller-manager-kubenode01 kube-system kube-controller-manager-kubenode02 kube-system kube-controller-manager-kubenode03 kube-system kube-flannel-dwpxs kube-system kube-flannel-jdtgg kube-system kube-flannel-lv8kw kube-system kube-proxy-4k4qz kube-system kube-proxy-4njtd kube-system kube-proxy-dlhcc kube-system kube-scheduler-kubenode01 kube-system kube-scheduler-kubenode02 kube-system kube-scheduler-kubenode03 kube-system nodelocaldns-gfvb7 kube-system nodelocaldns-jr6ft kube-system nodelocaldns-vwhzg wide --all-namespaces
READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
0/1 Completed 0 5m19s 10.233.65.23 kubenode02
1/1 Running 4 10d 10.233.64.71 kubenode01
0/1 Init:0/1 3 5m14s 10.233.65.24 kubenode02
1/1 Running 0 66m 10.233.65.19 kubenode02
1/1 Running 0 66m 10.233.66.2 kubenode03
2/2 Running 8 10d 10.233.64.74 kubenode01
2/2 Running 8 10d 10.233.64.72 kubenode01
1/1 Running 4 10d 10.233.64.73 kubenode01
1/1 Running 16 10d 10.233.65.18 kubenode02
1/1 Running 4 10d 10.233.64.70 kubenode01
1/1 Running 4 10d 10.233.64.75 kubenode01
1/1 Running 4 10d 172.16.25.80 kubenode01
1/1 Running 18 7d5h 172.16.25.83 kubenode02
1/1 Running 15 7d5h 172.16.25.97 kubenode03
1/1 Running 0 10d 172.16.25.80 kubenode01
1/1 Running 0 7d5h 172.16.25.83 kubenode02
1/1 Running 0 7d5h 172.16.25.97 kubenode03
1/1 Running 4 10d 172.16.25.80 kubenode01
1/1 Running 16 7d5h 172.16.25.83 kubenode02
1/1 Running 13 7d5h 172.16.25.97 kubenode03
1/1 Running 0 3h34m 172.16.25.80 kubenode01
1/1 Running 0 3h34m 172.16.25.83 kubenode02
1/1 Running 0 3h34m 172.16.25.97 kubenode03
1/1 Running 0 10d 172.16.25.80 kubenode01
1/1 Running 0 7d5h 172.16.25.83 kubenode02
1/1 Running 0 7d5h 172.16.25.97 kubenode03
1/1 Running 15 7d5h 172.16.25.97 kubenode03
1/1 Running 16 7d5h 172.16.25.83 kubenode02
1/1 Running 4 10d 172.16.25.80 kubenode01

bash-4.4# cat /proc/cpuinfo | grep processor | wc -l
4
bash-4.4#
bash-4.4# free -m
total used free shared buff/cache available
Mem: 16013 579 12578 1 2855 15135
Swap: 2047 0 2047
bash-4.4#

bash-4.4# helm upgrade --install wire-server wire/wire-server -f my-wire-server/values.yaml -f my-wire-server/secrets.yaml --wait --debug
history.go:56: [debug] getting history for release wire-server
upgrade.go:123: [debug] preparing upgrade for wire-server
upgrade.go:131: [debug] performing update for wire-server
upgrade.go:303: [debug] creating upgraded release for wire-server
client.go:282: [debug] Starting delete for "cassandra-migrations" Job
client.go:122: [debug] creating 1 resource(s)
client.go:491: [debug] Watching for changes to Job cassandra-migrations with timeout of 5m0s
client.go:519: [debug] Add/Modify event for cassandra-migrations: ADDED
client.go:558: [debug] cassandra-migrations: Jobs active: 0, jobs failed: 0, jobs succeeded: 0
client.go:519: [debug] Add/Modify event for cassandra-migrations: MODIFIED
client.go:558: [debug] cassandra-migrations: Jobs active: 1, jobs failed: 0, jobs succeeded: 0
client.go:519: [debug] Add/Modify event for cassandra-migrations: MODIFIED
client.go:282: [debug] Starting delete for "elasticsearch-index-create" Job
client.go:122: [debug] creating 1 resource(s)
client.go:491: [debug] Watching for changes to Job elasticsearch-index-create with timeout of 5m0s
client.go:519: [debug] Add/Modify event for elasticsearch-index-create: ADDED
client.go:558: [debug] elasticsearch-index-create: Jobs active: 0, jobs failed: 0, jobs succeeded: 0
client.go:519: [debug] Add/Modify event for elasticsearch-index-create: MODIFIED
client.go:558: [debug] elasticsearch-index-create: Jobs active: 1, jobs failed: 0, jobs succeeded: 0
upgrade.go:369: [debug] warning: Upgrade "wire-server" failed: pre-upgrade hooks failed: timed out waiting for the condition
Error: UPGRADE FAILED: pre-upgrade hooks failed: timed out waiting for the condition
helm.go:81: [debug] pre-upgrade hooks failed: timed out waiting for the condition
UPGRADE FAILED
main.newUpgradeCmd.func2
helm.sh/helm/v3/cmd/helm/upgrade.go:157
github.com/spf13/cobra.(*Command).execute
github.com/spf13/cobra@v1.1.1/command.go:850
github.com/spf13/cobra.(*Command).ExecuteC
github.com/spf13/cobra@v1.1.1/command.go:958
github.com/spf13/cobra.(*Command).Execute
github.com/spf13/cobra@v1.1.1/command.go:895
main.main
helm.sh/helm/v3/cmd/helm/helm.go:80
runtime.main
runtime/proc.go:225
runtime.goexit
runtime/asm_amd64.s:1371

bash-4.4# helm list --all
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
cassandra-external default 2 2021-06-20 11:43:21.448671888 +0000 UTC deployed cassandra-external-2.108.0
databases-ephemeral default 2 2021-06-20 11:43:41.528915369 +0000 UTC deployed databases-ephemeral-2.108.0
elasticsearch-external default 2 2021-06-20 11:43:33.670740997 +0000 UTC deployed elasticsearch-external-2.108.0
fake-aws default 2 2021-06-20 11:49:41.841649782 +0000 UTC deployed fake-aws-2.108.0
minio-external default 4 2021-06-20 11:45:08.514165984 +0000 UTC deployed minio-external-2.108.0
smtp default 2 2021-06-20 11:54:14.683598062 +0000 UTC deployed demo-smtp-2.108.0 1.0
wire-server default 9 2021-06-20 12:50:43.972757468 +0000 UTC failed wire-server-2.108.0

JonnyX11 · 2021-06-23T07:53:36Z

Hello @JonnyX11

we are always happy to help, but we kindly ask people who reach out for help to be cooperative. So, would you mind following the template structure that we put in place for these kind of scenarios? You might notice some significant overlap between the questions lisyted in the template and the ones asked by @arthurwolf

Additionally to what is in the template, please provide us with some more context. For example, is it your first installation or are you trying to upgrade an installation? If the latter, did you check the release notes for the new version you installed - also for wire-server? Please note, you will find the code for each chart in there.

Lastly, please provide us with the output of kubectl get pods -o wide --all-namespaces.

Thank you.

The details was OK on the last post ? it was clear ?

JonnyX11 · 2021-07-01T12:51:10Z

Anyone?

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

wire-server installation failed #470

wire-server installation failed #470

JonnyX11 commented Jun 13, 2021

arthurwolf commented Jun 13, 2021 via email

JonnyX11 commented Jun 14, 2021

JonnyX11 commented Jun 15, 2021 •

edited

JonnyX11 commented Jun 16, 2021

arthurwolf commented Jun 17, 2021

arthurwolf commented Jun 17, 2021

lucendio commented Jun 18, 2021

JonnyX11 commented Jun 20, 2021 •

edited

JonnyX11 commented Jun 23, 2021 •

edited

JonnyX11 commented Jul 1, 2021

wire-server installation failed #470

wire-server installation failed #470

Comments

JonnyX11 commented Jun 13, 2021

arthurwolf commented Jun 13, 2021 via email

JonnyX11 commented Jun 14, 2021

JonnyX11 commented Jun 15, 2021 • edited

JonnyX11 commented Jun 16, 2021

arthurwolf commented Jun 17, 2021

arthurwolf commented Jun 17, 2021

lucendio commented Jun 18, 2021

JonnyX11 commented Jun 20, 2021 • edited

Basic information

How did you install Wire?

JonnyX11 commented Jun 23, 2021 • edited

JonnyX11 commented Jul 1, 2021

JonnyX11 commented Jun 15, 2021 •

edited

JonnyX11 commented Jun 20, 2021 •

edited

JonnyX11 commented Jun 23, 2021 •

edited