Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Default instance type t2.medium insufficient for demo repo #20

Open
soulshake opened this issue Oct 19, 2020 · 0 comments
Open

Default instance type t2.medium insufficient for demo repo #20

soulshake opened this issue Oct 19, 2020 · 0 comments

Comments

@soulshake
Copy link
Contributor

Deployments always fail as follows:

+ skaffold deploy -p ephemeral-deployment -f skaffold.yaml --build-artifacts=default-tags.json
Tags used in deployment:
 - soulshake/front-end -> soulshake/front-end:main
Starting deploy...
WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: /home/runner/.kube/config
WARNING: Kubernetes configuration file is world-readable. This is insecure. Location: /home/runner/.kube/config
WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: /home/runner/.kube/config
WARNING: Kubernetes configuration file is world-readable. This is insecure. Location: /home/runner/.kube/config
Release "microservice-demo-pr-front-end-2" has been upgraded. Happy Helming!
NAME: microservice-demo-pr-front-end-2
LAST DEPLOYED: Sat Oct 17 17:05:12 2020
NAMESPACE: pr-front-end-2
STATUS: deployed
REVISION: 2
TEST SUITE: None
Waiting for deployments to stabilize...
 - pr-front-end-2:deployment/carts-db is ready. [12/13 deployment(s) still pending]
 - pr-front-end-2:deployment/queue-master: running [kubectl --context ephemeral-demo-spot rollout status deployment queue-master --namespace pr-front-end-2 --watch=false]
 - stdout: ""
 - stderr: "error: deployment \"queue-master\" exceeded its progress deadline\n"
 - cause: exit status 1
 - pr-front-end-2:deployment/queue-master failed. Error: running [kubectl --context ephemeral-demo-spot rollout status deployment queue-master --namespace pr-front-end-2 --watch=false]
 - stdout: ""
 - stderr: "error: deployment \"queue-master\" exceeded its progress deadline\n"
 - cause: exit status 1.
12/13 deployment(s) failed
 - pr-front-end-2:deployment/carts: running [kubectl --context ephemeral-demo-spot rollout status deployment carts --namespace pr-front-end-2 --watch=false]
 - stdout: ""
 - stderr: "error: deployment \"carts\" exceeded its progress deadline\n"
 - cause: exit status 1
 - pr-front-end-2:deployment/carts failed. Error: running [kubectl --context ephemeral-demo-spot rollout status deployment carts --namespace pr-front-end-2 --watch=false]
 - stdout: ""
 - stderr: "error: deployment \"carts\" exceeded its progress deadline\n"
 - cause: exit status 1.
Error: Process completed with exit code 1.

This seems to be because t2.medium instances can run up to 17 pods. The demo project has 13; these plus the services of the ephemeral.run repo exceed the maximum, so new pods remain forever in Pending, causing the deployment to time out and fail.

It would be nice to check for this situation or document workarounds (add a node, increase the desired ASG size, change default instance type, etc).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant