Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add initial execution progress streaming impl #6544

Merged
merged 3 commits into from May 13, 2024
Merged

Conversation

bduffany
Copy link
Member

@bduffany bduffany commented May 9, 2024

  • Adds a Publisher interface that lets us publish the new progress updates concurrently with other progress updates. We need this because there's a loop in ExecuteTaskAndStreamResults which periodically publishes progress updates while the task is executing, but we also want to publish fine-grained progress events while this loop is ongoing.
  • Publish the basic progress updates: pulling image, downloading inputs, executing, uploading outputs.
  • Update bb execute to log debug messages with these progress updates.
  • Some small refactoring in the rexec / operation packages.

TODO in future PRs:

  • Publish progress for VM boot / snapshot resume.
  • Don't publish "pulling image" status if we're not actually pulling an image.

Screenshot showing it in action (with bb execute for now as a simple client):

rec-2024-05-09_18.28.59.mp4

Related issues: N/A

@bduffany bduffany force-pushed the exec-progress-impl branch 2 times, most recently from 69a1a16 to c7630cf Compare May 9, 2024 22:51

if stage == repb.ExecutionStage_COMPLETED {
operation.Done = true
if er != nil {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

uber-nit: Could you rename er? It looks like a typo of err

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

haha it does 😆 done

@bduffany bduffany merged commit 3239668 into master May 13, 2024
18 checks passed
@bduffany bduffany deleted the exec-progress-impl branch May 13, 2024 21:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants