Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add cargo risczero benchmark command #1302

Open
wants to merge 29 commits into
base: main
Choose a base branch
from

Conversation

Cardosaum
Copy link
Contributor

@Cardosaum Cardosaum commented Jan 8, 2024

Closes #1301

@Cardosaum Cardosaum self-assigned this Jan 8, 2024
Copy link

vercel bot commented Jan 8, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
website ✅ Ready (Inspect) Visit Preview 💬 Add feedback Jan 25, 2024 3:56am

@Cardosaum Cardosaum added the enhancement New feature or request label Jan 8, 2024
@@ -33,13 +33,17 @@ reqwest-middleware = "0.2"
reqwest-retry = "0.3"
risc0-build = { workspace = true }
risc0-r0vm = { workspace = true, optional = true }
risc0-zkvm = { workspace = true, optional = true }
risc0-zkvm = { workspace = true, default-features = false, features = ["std", "getrandom"] }
risc0-zkvm-methods = { workspace = true, default-features = false }
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs to be dropped, it can't be published

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes!

I was thinking in moving the dependencies we have on risc0-zkvm-methods to a new crate risc0-zkvm-methods-core that we could publish, this way we would solve the issue. That way both risc0-zkvm-methods and cargo-risczero could depend on risc0-zkvm-methods-core.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think any of the benchmark binaries need any 'core' crates because the interface is very simple. We pass in the number of iterations to run, so there's no types that need to be shared between host and guest.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh I see... I followed this approach because this would be similar to what we do under /examples having a core that can be used both in guest and host.

I did the change before seeing this comment. Please let me know if you'd like to revert the new risc0-zkvm-methods-core and I'll change it 🙇‍♂️

risc0/build/src/lib.rs Outdated Show resolved Hide resolved
@flaub flaub changed the title feat: Add cargo risczero benchmark command Add cargo risczero benchmark command Jan 23, 2024
@flaub flaub added this to the 0.21.0 milestone Jan 23, 2024
Move code from `risc0-zkvm-methods` to the new crate
Also remove all code responsible to download a elf binary from GitHub, replacing it with an `include_bites!` for the elf binary previously generated by xtask.
@Cardosaum
Copy link
Contributor Author

@flaub , I tried to copy your logic from gen-receipts and apply the same here so we can include the elf binary in compile time.

I have one question on how we ensure that the file generated by xtask will be present when we build cargo-risczero as now we have a dependency on running xtask before being able to compile cargo-risczero.

Would you have some direction on that?

Also, the whole logic for doing the benchmarks got way simpler this way, thanks for letting me know there was this option for doing it at compile time! ^^

Comment on lines +40 to +59
mod benchmark {
pub fn build_benchmark_elf(path: &str) {
let benchmark_elf = std::env::current_dir().unwrap().join(path);
if let Ok(content) = std::fs::read(&benchmark_elf) {
std::fs::write(&benchmark_elf, content).unwrap();
} else {
build_elf(benchmark_elf.to_str().unwrap());
}
}

fn build_elf(path: &str) {
let is_success = std::process::Command::new("cargo")
.args(["xtask", "gen-benchmark", "--path", path])
.status()
.unwrap()
.success();
assert!(is_success);
}
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I might have found a way to compile it automatically during build time! 😄

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Giving credit where credit is due, I found it already in our code base :)

@Cardosaum Cardosaum marked this pull request as ready for review January 24, 2024 21:16
@Cardosaum Cardosaum requested a review from flaub January 24, 2024 21:33
Copy link
Contributor

@SchmErik SchmErik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me - I think it would be good for @flaub to review and approve before merging

Copy link
Member

@flaub flaub left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I'll take a pass at this at some point to get it landed

@nategraf nategraf removed their request for review February 15, 2024 21:33
@SchmErik SchmErik assigned SchmErik and unassigned SchmErik Mar 5, 2024
@flaub flaub assigned flaub and unassigned Cardosaum Apr 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Feature] Add cargo risczero benchmark command
3 participants