Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI Flake: Dualstack jobs failing due to running out of disk space #4287

Open
tssurya opened this issue Apr 18, 2024 · 1 comment
Open

CI Flake: Dualstack jobs failing due to running out of disk space #4287

tssurya opened this issue Apr 18, 2024 · 1 comment
Assignees
Labels
kind/ci-flake Flakes seen in CI

Comments

@tssurya
Copy link
Member

tssurya commented Apr 18, 2024

See
https://github.com/ovn-org/ovn-kubernetes/actions/runs/8728901846/job/23968836196?pr=4245
https://github.com/ovn-org/ovn-kubernetes/actions/runs/8728901846/job/23968836684?pr=4245

~/work/ovn-kubernetes/ovn-kubernetes/test/e2e ~/work/ovn-kubernetes/ovn-kubernetes/test
+ go test -test.timeout 180m -v . -ginkgo.v -ginkgo.focus DualStack -ginkgo.timeout 3h -ginkgo.flake-attempts 2 '-ginkgo.skip=Should validate flow data of br-int is sent to an external gateway with sflow|ipv6 pod|Should validate connectivity before and after deleting all the db-pods at once in Non-HA mode|  e2e br-int NetFlow export validation|Should ensure load balancer service|LGW|disabling egress nodes impeding Legacy health check|Multi Homing|Node IP and MAC address migration|Multi node zones interconnect|External Gateway|Kubevirt Virtual Machines' -provider skeleton -kubeconfig /home/runner/ovn.conf --num-nodes=2 --report-dir=/home/runner/work/ovn-kubernetes/ovn-kubernetes/test/_artifacts --report-prefix=DualStack-conversion-shared-HA-interconnect-disabled_
# github.com/ovn-org/ovn-kubernetes/test/e2e.test
/opt/hostedtoolcache/go/1.21.9/x64/pkg/tool/linux_amd64/link: mapping output file failed: no space left on device
FAIL	github.com/ovn-org/ovn-kubernetes/test/e2e [build failed]
FAIL
make: *** [Makefile:32: control-plane] Error 1
make: Leaving directory '/home/runner/work/ovn-kubernetes/ovn-kubernetes/test'
Error: Process completed with exit code 2.
@tssurya tssurya added the kind/ci-flake Flakes seen in CI label Apr 18, 2024
@jluhrsen jluhrsen self-assigned this Apr 23, 2024
jluhrsen added a commit to jluhrsen/ovn-kubernetes that referenced this issue Apr 23, 2024
jluhrsen added a commit to jluhrsen/ovn-kubernetes that referenced this issue Apr 23, 2024
jluhrsen added a commit to jluhrsen/ovn-kubernetes that referenced this issue Apr 24, 2024
ISSUE: ovn-org#4287
JIRA: https://issues.redhat.com/browse/SDN-4786

Signed-off-by: Jamo Luhrsen <jluhrsen@gmail.com>
@jluhrsen
Copy link
Contributor

@tssurya @trozet , I looked through all the CI runs in the last 5 pages and did not find this happening. I assume it
must not be very frequent. Here is a PR that will add some output before and after go test for those jobs you saw
this in. That will at least give us a little data to check out the next time this happens. I'll try to actively monitor CI to
find when it happens again.

jluhrsen added a commit to jluhrsen/ovn-kubernetes that referenced this issue Apr 25, 2024
ISSUE: ovn-org#4287
JIRA: https://issues.redhat.com/browse/SDN-4786

Signed-off-by: Jamo Luhrsen <jluhrsen@gmail.com>
jluhrsen added a commit to jluhrsen/ovn-kubernetes that referenced this issue Apr 25, 2024
ISSUE: ovn-org#4287
JIRA: https://issues.redhat.com/browse/SDN-4786

Signed-off-by: Jamo Luhrsen <jluhrsen@gmail.com>
jluhrsen added a commit to jluhrsen/ovn-kubernetes that referenced this issue Apr 29, 2024
ISSUE: ovn-org#4287
JIRA: https://issues.redhat.com/browse/SDN-4786

Signed-off-by: Jamo Luhrsen <jluhrsen@gmail.com>
jluhrsen added a commit to jluhrsen/ovn-kubernetes that referenced this issue Apr 30, 2024
ISSUE: ovn-org#4287
JIRA: https://issues.redhat.com/browse/SDN-4786

Signed-off-by: Jamo Luhrsen <jluhrsen@gmail.com>
jluhrsen added a commit to jluhrsen/ovn-kubernetes that referenced this issue May 1, 2024
ISSUE: ovn-org#4287
JIRA: https://issues.redhat.com/browse/SDN-4786

Signed-off-by: Jamo Luhrsen <jluhrsen@gmail.com>
jluhrsen added a commit to jluhrsen/ovn-kubernetes that referenced this issue May 1, 2024
ISSUE: ovn-org#4287
JIRA: https://issues.redhat.com/browse/SDN-4786

Signed-off-by: Jamo Luhrsen <jluhrsen@gmail.com>
jluhrsen added a commit to jluhrsen/ovn-kubernetes that referenced this issue May 2, 2024
ISSUE: ovn-org#4287
JIRA: https://issues.redhat.com/browse/SDN-4786

Signed-off-by: Jamo Luhrsen <jluhrsen@gmail.com>
jluhrsen added a commit to jluhrsen/ovn-kubernetes that referenced this issue May 2, 2024
ISSUE: ovn-org#4287
JIRA: https://issues.redhat.com/browse/SDN-4786

Signed-off-by: Jamo Luhrsen <jluhrsen@gmail.com>
jluhrsen added a commit to jluhrsen/ovn-kubernetes that referenced this issue May 2, 2024
ISSUE: ovn-org#4287
JIRA: https://issues.redhat.com/browse/SDN-4786

Signed-off-by: Jamo Luhrsen <jluhrsen@gmail.com>
jluhrsen added a commit to jluhrsen/ovn-kubernetes that referenced this issue May 3, 2024
ISSUE: ovn-org#4287
JIRA: https://issues.redhat.com/browse/SDN-4786

Signed-off-by: Jamo Luhrsen <jluhrsen@gmail.com>
jluhrsen added a commit to jluhrsen/ovn-kubernetes that referenced this issue May 3, 2024
ISSUE: ovn-org#4287
JIRA: https://issues.redhat.com/browse/SDN-4786

Signed-off-by: Jamo Luhrsen <jluhrsen@gmail.com>
jluhrsen added a commit to jluhrsen/ovn-kubernetes that referenced this issue May 3, 2024
ISSUE: ovn-org#4287
JIRA: https://issues.redhat.com/browse/SDN-4786

Signed-off-by: Jamo Luhrsen <jluhrsen@gmail.com>
jluhrsen added a commit to jluhrsen/ovn-kubernetes that referenced this issue May 3, 2024
ISSUE: ovn-org#4287
JIRA: https://issues.redhat.com/browse/SDN-4786

Signed-off-by: Jamo Luhrsen <jluhrsen@gmail.com>
@trozet trozet changed the title CI Flake: Dualstack jobs failing due to running out of memory CI Flake: Dualstack jobs failing due to running out of disk space May 9, 2024
trozet pushed a commit that referenced this issue May 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/ci-flake Flakes seen in CI
Projects
None yet
Development

No branches or pull requests

2 participants