Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

WIP PERFSCALE-2860 - Re-enable jobs on new schedule balancing #51420

Draft
wants to merge 4 commits into
base: master
Choose a base branch
from
Draft

WIP PERFSCALE-2860 - Re-enable jobs on new schedule balancing #51420

wants to merge 4 commits into from
+5,208 −4,718

Conversation

afcollins
Copy link
Contributor

Restoring perfscale jobs on renewed, balanced schedule.

@openshift-ci openshift-ci bot requested review from memodi and mffiedler April 26, 2024 23:20
@openshift-ci OpenShift CI
Copy link
Contributor

openshift-ci bot commented Apr 26, 2024

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: afcollins
Once this PR has been reviewed and has the lgtm label, please assign qiliredhat for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-merge-robot openshift-merge-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Apr 29, 2024
@openshift-ci-robot
Copy link
Contributor

@afcollins, pj-rehearse: unable to determine affected jobs. This could be due to a branch that needs to be rebased. ERROR:

couldn't prepare candidate: couldn't rebase candidate onto master: %!w(<nil>)
Interacting with pj-rehearse

Comment: /pj-rehearse to run up to 5 rehearsals
Comment: /pj-rehearse skip to opt-out of rehearsals
Comment: /pj-rehearse {test-name}, with each test separated by a space, to run one or more specific rehearsals
Comment: /pj-rehearse more to run up to 10 rehearsals
Comment: /pj-rehearse max to run up to 25 rehearsals
Comment: /pj-rehearse auto-ack to run up to 5 rehearsals, and add the rehearsals-ack label on success
Comment: /pj-rehearse abort to abort all active rehearsals

Once you are satisfied with the results of the rehearsals, comment: /pj-rehearse ack to unblock merge. When the rehearsals-ack label is present on your PR, merge will no longer be blocked by rehearsals.
If you would like the rehearsals-ack label removed, comment: /pj-rehearse reject to re-block merging.

@openshift-merge-robot openshift-merge-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Apr 29, 2024
@openshift-ci OpenShift CI
Copy link
Contributor

openshift-ci bot commented Apr 29, 2024

@afcollins: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@afcollins
Copy link
Contributor Author

Google sheet generated from the configurations here.

@afcollins
Copy link
Contributor Author

cc: @qiliRedHat @rpattath

@openshift-merge-robot openshift-merge-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 5, 2024
@afcollins afcollins changed the title PERFSCALE-2860 - Re-enable jobs on new schedule balancing WIP PERFSCALE-2860 - Re-enable jobs on new schedule balancing May 9, 2024
@afcollins afcollins marked this pull request as draft May 9, 2024 18:28
@openshift-ci openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label May 9, 2024
@openshift-merge-robot openshift-merge-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 10, 2024
@openshift-ci-robot
Copy link
Contributor

[REHEARSALNOTIFIER]
@afcollins: the pj-rehearse plugin accommodates running rehearsal tests for the changes in this PR. Expand 'Interacting with pj-rehearse' for usage details. The following rehearsable tests have been affected by this change:

Test name Repo Type Reason
pull-ci-openshift-qe-ocp-qe-perfscale-ci-main-aws-4.16-nightly-x86-cilium-control-plane-120nodes openshift-qe/ocp-qe-perfscale-ci presubmit Presubmit changed
pull-ci-openshift-qe-ocp-qe-perfscale-ci-main-aws-4.16-nightly-x86-cilium-control-plane-24nodes openshift-qe/ocp-qe-perfscale-ci presubmit Presubmit changed
pull-ci-openshift-qe-ocp-qe-perfscale-ci-main-aws-4.16-nightly-x86-cilium-data-path-9nodes openshift-qe/ocp-qe-perfscale-ci presubmit Presubmit changed
pull-ci-openshift-qe-ocp-qe-perfscale-ci-main-aws-4.16-nightly-x86-conc-builds-3nodes openshift-qe/ocp-qe-perfscale-ci presubmit Presubmit changed
pull-ci-openshift-qe-ocp-qe-perfscale-ci-main-aws-4.16-nightly-x86-payload-node-density-cni-6nodes openshift-qe/ocp-qe-perfscale-ci presubmit Presubmit changed
pull-ci-openshift-qe-ocp-qe-perfscale-ci-main-aws-4.16-nightly-x86-payload-node-density-heavy-6nodes openshift-qe/ocp-qe-perfscale-ci presubmit Presubmit changed
pull-ci-openshift-qe-ocp-qe-perfscale-ci-main-rosa-4.15-nightly-x86-cluster-density-v2-249nodes openshift-qe/ocp-qe-perfscale-ci presubmit Presubmit changed
pull-ci-openshift-qe-ocp-qe-perfscale-ci-main-rosa-4.15-nightly-x86-conc-builds-3nodes openshift-qe/ocp-qe-perfscale-ci presubmit Presubmit changed
pull-ci-openshift-qe-ocp-qe-perfscale-ci-main-rosa-4.15-nightly-x86-node-density-cni-24nodes openshift-qe/ocp-qe-perfscale-ci presubmit Presubmit changed
pull-ci-openshift-qe-ocp-qe-perfscale-ci-main-rosa_hcp-4.15-nightly-x86-node-density-cni-24nodes openshift-qe/ocp-qe-perfscale-ci presubmit Presubmit changed
periodic-ci-openshift-qe-ocp-qe-perfscale-ci-main-rosa_hcp-4.15-nightly-x86-data-path-9nodes N/A periodic Ci-operator config changed
periodic-ci-openshift-qe-ocp-qe-perfscale-ci-main-aws-4.14-nightly-x86-node-density-heavy-24nodes N/A periodic Periodic changed
periodic-ci-openshift-qe-ocp-qe-perfscale-ci-main-aws-4.15-nightly-x86-node-density-24nodes-ovn-fips N/A periodic Periodic changed
periodic-ci-openshift-qe-ocp-qe-perfscale-ci-main-rosa_hcp-4.14-stable-x86-loaded-upgrade-from-4.14-perfscale-rosa-hcp-multiaz-24nodes-stage-loaded-upgrade414 N/A periodic Periodic changed
periodic-ci-openshift-qe-ocp-qe-perfscale-ci-main-aws-4.15-nightly-x86-node-density-heavy-24nodes N/A periodic Periodic changed
periodic-ci-openshift-qe-ocp-qe-perfscale-ci-main-rosa_hcp-4.14-nightly-x86-node-density-heavy-24nodes N/A periodic Periodic changed
periodic-ci-openshift-qe-ocp-qe-perfscale-ci-main-aws-4.13-nightly-x86-node-density-heavy-24nodes N/A periodic Periodic changed
periodic-ci-openshift-qe-ocp-qe-perfscale-ci-main-aws-4.15-nightly-x86-loaded-upgrade-from-4.14-perfscale-aws-24nodes-loaded-upgrade-414to415 N/A periodic Periodic changed
periodic-ci-openshift-qe-ocp-qe-perfscale-ci-main-rosa-4.14-nightly-x86-node-density-heavy-24nodes N/A periodic Periodic changed
periodic-ci-openshift-qe-ocp-qe-perfscale-ci-main-rosa-4.14-nightly-x86-control-plane-120nodes N/A periodic Periodic changed
periodic-ci-openshift-qe-ocp-qe-perfscale-ci-main-aws-4.15-nightly-x86-control-plane-24nodes N/A periodic Periodic changed
periodic-ci-openshift-qe-ocp-qe-perfscale-ci-main-rosa-4.15-nightly-x86-data-path-9nodes N/A periodic Ci-operator config changed
periodic-ci-openshift-qe-ocp-qe-perfscale-ci-main-azure-4.15-nightly-x86-data-path-9nodes N/A periodic Periodic changed
periodic-ci-openshift-qe-ocp-qe-perfscale-ci-main-aws-4.14-nightly-x86-control-plane-etcdencrypt-24nodes N/A periodic Periodic changed
periodic-ci-openshift-qe-ocp-qe-perfscale-ci-main-rosa-4.14-nightly-x86-control-plane-24nodes N/A periodic Periodic changed

A total of 64 jobs have been affected by this change. The above listing is non-exhaustive and limited to 25 jobs.

A full list of affected jobs can be found here

Interacting with pj-rehearse

Comment: /pj-rehearse to run up to 5 rehearsals
Comment: /pj-rehearse skip to opt-out of rehearsals
Comment: /pj-rehearse {test-name}, with each test separated by a space, to run one or more specific rehearsals
Comment: /pj-rehearse more to run up to 10 rehearsals
Comment: /pj-rehearse max to run up to 25 rehearsals
Comment: /pj-rehearse auto-ack to run up to 5 rehearsals, and add the rehearsals-ack label on success
Comment: /pj-rehearse abort to abort all active rehearsals

Once you are satisfied with the results of the rehearsals, comment: /pj-rehearse ack to unblock merge. When the rehearsals-ack label is present on your PR, merge will no longer be blocked by rehearsals.
If you would like the rehearsals-ack label removed, comment: /pj-rehearse reject to re-block merging.

@afcollins
Copy link
Contributor Author

Alright, I rebalanced the jobs with those 4.16 that have been added since.

I also split apart monthly and bi-weekly jobs such as not to overburden any particular week-long Jedi shift.

We have 5 workloads that all run on their own days, so each week contains roughly the same mix of each workload.

The "5 workloads" are: [ control-plane-24, control-plane-120, data-path, node-density-heavy, loaded-upgrade ]

The 'periodic' tab in this gsheet is updated with the changes from this PR. Please review.

You may find visual schedules for:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@memodi memodi Awaiting requested review from memodi

@mffiedler mffiedler Awaiting requested review from mffiedler

Assignees
No one assigned
Labels
do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress.
Projects
None yet
Milestone
No milestone
3 participants
@afcollins @openshift-ci-robot @openshift-merge-robot