The release-0.6 branch looks healthy https://testgrid.k8s.io/sig-scheduling#pull-kueue-test-integration-release-0-6&width=20

The only difference in the presubmit configuration is that main is running on golang 1.22

/assign @gabesaba

@alculquicondor: GitHub didn't allow me to assign the following users: gabesaba.

Note that only kubernetes-sigs members with read permissions, repo collaborators and people who have commented on this issue/PR can be assigned. Additionally, issues/PRs can only have 10 assignees at the same time.
For more information please see the contributor guide

Probably because junit output is too large - there's an error message that it is malformed since over 100MB

We could either bump this limit, or reduce verbosity

/assign @gabesaba

removing -v ginkgo flag from test-integration target, junit.xml went from 204MB to 88MB. This would fix issue, but we're still close to the limit.

Looking next into any particularly spammy logs

Much of the size can be attributed to a few tests. Below are the tests with output > 1MB (after HTML unescaping, so the actual size is larger)

within those tests, attributing output to certain lines

We're repeatedly reconciling unschedulable workloads without any backoff. Should there be a backoff here? I imagine that a backoff of even a fraction of a second would drastically reduce the logging output here

212MB to 15MB after changing scheduler.go:128 to 10ms

$ wc -c before.xml fix.xml 
211955097 before.xml
 14650361 fix.xml

Uhm.... maybe we can use a NewItemExponentialFailureRateLimiter to have a backoff when we couldn't admit any workload in that iteration. And we clear the backoff anytime we successfully admit something.