New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implement missing parts of OSS RE caching #477
base: main
Are you sure you want to change the base?
Conversation
The TODOs around symlinking are related to #222 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks so much for this, and the great testing you have done3. Two things I'm still figuring out:
- Why didn't you get a warn - I've noted where I expected that to pop out. Is that line not being hit for you?
- Still figuring out if there are internal implications for upload blob taking a digest, or if it should have done so all along.
is_topologically_sorted: false, | ||
}) | ||
})?; | ||
anyhow::Ok(ActionResult { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Any reason to not just Ok
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's just splatter from copy pasting the function below and inverting every operation, instead of coding it from scratch. I'll tidy this up, a few of these don't even need to be Result.
|| !action_result.output_file_symlinks.is_empty() | ||
|| !action_result.output_directory_symlinks.is_empty() | ||
{ | ||
anyhow::bail!( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We tend to avoid bail because it can be hard to reason about - easier to see the explicit return
@@ -91,6 +91,9 @@ pub struct TSubsysPerfCount { | |||
pub struct TActionResult2 { | |||
pub output_files: Vec<TFile>, | |||
pub output_directories: Vec<TDirectory2>, | |||
// TODO: output_symlinks (use in preference when output_paths mode is used the execution side) | |||
// TODO: output_file_symlinks (deprecated) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why mark this as a todo? Is it something we should add? Or is this just for when servers return deprecated stuff?
@@ -1141,6 +1143,9 @@ impl RemoteExecutionClientImpl { | |||
..Default::default() | |||
}, | |||
) | |||
.inspect_err(|err| { | |||
tracing::warn!("write_action_result failed: {err}"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does this not appear in https://github.com/facebook/buck2/blob/main/app/buck2_execute_impl/src/executors/caching.rs#L108C25-L108C25 as a warning? It certainly should
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That line may have run, but I definitely did not see it in the terminal. Thanks for the pointer, I will have a look at why.
Having upload blob take a digest shouldn't be a big deal since I think the amount of data flowing through this API isn't big enough to matter. I can see why this is more convenient and matches the upload API better. However, once we are going through that path, maybe we should just delete |
There is a lot of "code overhead" for each piece of GRPC client API you want to expose. There are about 5 layers of nesting that, for |
Some of these layers of forwarding are so we can inject the internal closed source RE client (closed source because it uses a lot of systems that aren't open source, rather than because it has magical properties). They might need to stay (or might not). |
Is there anything I can do here to help push this to being landed? I'm happy to finish the PR if you'd like @cormacrelf |
Hi @benbrittain, I haven't had any time to work on this recently. I've given you push access to my fork cormacrelf/buck2, so you can keep working on the rbe-writes branch. |
@cormacrelf @benbrittain any update on this PR? This might help our RE execution efficiency, and I can help if no one is working on it. |
Hey @miaoyipu! It's on my near term TODO but I'm certainly not against you picking it up. If you do so though, let me know :) |
Hi @cormacrelf
I am also trying to use this with Buildbuddy and disabled \edit: Never mind, it's working now. Thank you for the implementation @cormacrelf! |
This enables action caching on OSS builds. There were two missing pieces:
internal
is the right one for it to be in, or when these tests get run.I've successfully used this implementation with BuildBuddy, both from Linux (using remote_enabled = True), and from macOS, with remote_enabled = False but caching still enabled. I have only tested it with RBE v2.3+ output_paths mode.
As an implementation note, I found it very difficult to even figure out that these parts weren't implemented. Buck2 was just saying "0% cache hits mate" and reporting that every
re_action_cache
(= attempts to fetch a cached action, not attempts to cache an executed action!) had failed. None of these failures were logged anywhere. I don't know what the appropriate mechanism to log this stuff is, but I whacked a tracing::warn for failed action result cache writes in there.