Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[release-automation] Separate steps into groups #45355

Closed
wants to merge 3 commits into from

Conversation

khluu
Copy link
Contributor

@khluu khluu commented May 15, 2024

  • Apparently the block step blocks all the steps after it (unless the step specifies depends_on). This happens for mac sanity check step since it has no dependency: https://buildkite.com/ray-project/release-automation/builds/470/dag
  • This separates each platform sanity check to separate groups so they don't block each other
  • Also separate upload wheels step into its own group to allow more flexibility (upload to PyPI & TestPyPI doesn't have to be in the same build)
  • Fix typo forge-arm64 -> forge_arm64

khluu added 2 commits May 15, 2024 02:25
p
Signed-off-by: khluu <51931015+khluu@users.noreply.github.com>
Signed-off-by: khluu <51931015+khluu@users.noreply.github.com>
@khluu
Copy link
Contributor Author

khluu commented May 15, 2024

I tested with this build and was able to see each of the group start "waiting for agent" right after unblocking without waiting for other steps: https://buildkite.com/ray-project/release-automation/builds/476#_

Signed-off-by: khluu <51931015+khluu@users.noreply.github.com>
@aslonnie
Copy link
Collaborator

isn't it easier to just add the depends_on for steps? rather than breaking it down into pieces?

Copy link
Collaborator

@aslonnie aslonnie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

isn't it easier to just add the depends_on for steps? rather than breaking it down into pieces?

like each group now only contains 1 or 2 command steps.

@khluu
Copy link
Contributor Author

khluu commented May 15, 2024

isn't it easier to just add the depends_on for steps? rather than breaking it down into pieces?

for example, mac sanity checks don't really have any depends_on. I tried having it depends on the block step but that didn't really work.

@khluu khluu requested a review from aslonnie May 15, 2024 19:01
@aslonnie
Copy link
Collaborator

I tried having it depends on the block step but that didn't really work.

why? that is how it supposed to work?

Copy link
Collaborator

@aslonnie aslonnie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't really understand why using depends_on key does not work here..

@khluu
Copy link
Contributor Author

khluu commented May 16, 2024

@aslonnie You can take a look at this commit: 30fdfe8 which I set mac sanity check step to depend on its block step. The block step is blocked by the step before it though for some reason: https://buildkite.com/ray-project/release-automation/builds/474/dag

When I unblocked the step, the mac sanity check steps didn't even run or wait for agent .. they were just stuck there

@khluu khluu requested a review from aslonnie May 16, 2024 07:53
@aslonnie
Copy link
Collaborator

When I unblocked the step, the mac sanity check steps didn't even run or wait for agent .. they were just stuck there

you need to add an empty depends_on for the wait/block step too. otherwise, a wait/block step depends on all steps before it.

Copy link
Collaborator

@aslonnie aslonnie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

like this:

#45425

@khluu khluu closed this May 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants