Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Upload/download content failures during pod rollouts #978

Open
git-hyagi opened this issue Jun 20, 2023 · 0 comments
Open

[BUG] Upload/download content failures during pod rollouts #978

git-hyagi opened this issue Jun 20, 2023 · 0 comments
Labels

Comments

@git-hyagi
Copy link
Collaborator

Describe the bug
Our pods are not getting terminated in a graceful way. An image version upgrade or any change that requires deploying new pods will cause a rollout of the Deployments and the pods will be terminated even in the middle of a running task (download/upload content).

To Reproduce
Steps to reproduce the behavior:
Install the operator with 2 api and content replicas:

spec:
  api:
    replicas: 2
  content:
    replicas: 2
  image_version: latest
  image_web_version: latest

start to upload a file:

pulp file content upload --file Fedora-Workstation-Live-x86_64-35-1.2.iso --relative-path Fedora-Workstation-Live-x86_64-35-1.2.iso

while the upload is running, modify Pulp CR to "force" the redeploy of pods:

spec:
  image_version: stable
  image_web_version: stable

pulp client will receive the following error:

.uploads_update : put https://example-pulp.apps-crc.testing/pulp/api/v3/uploads/0188c54f-6ab8-7c75-9f6c-5014d0e0670b/
Response: 200
.uploads_update : put https://example-pulp.apps-crc.testing/pulp/api/v3/uploads/0188c54f-6ab8-7c75-9f6c-5014d0e0670b/
uploads_delete : delete https://example-pulp.apps-crc.testing/pulp/api/v3/uploads/0188c54f-6ab8-7c75-9f6c-5014d0e0670b/
Response: 204
Error: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

Expected behavior
The pods should not be terminated if there is/are any task(s) running.

Additional context
Verify which signal kubelet sends to the process in these scenarios.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants