Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add cargo risczero benchmark command #1302

Open
wants to merge 29 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
3cf5675
feat: Add `cargo risczero benchmark` command
Cardosaum Jan 8, 2024
4d9460b
Merge branch 'main' into cardosaum/add-cargo-risczero-benchmark
Cardosaum Jan 8, 2024
81515a7
chore: format and sort dependencies
Cardosaum Jan 8, 2024
5e90ce1
Refactor benchmark command to handle iterations
Cardosaum Jan 10, 2024
11c9cbf
Add risc-zkvm-methods crate to workspace
Cardosaum Jan 12, 2024
6baaca7
feat(cargo-build): Add ability to compile specified binaries.
Cardosaum Jan 22, 2024
f37a441
Add tracing, sha2, and serde_json dependencies
Cardosaum Jan 22, 2024
e13ac03
Add logic to download specified ELF binary from GitHub
Cardosaum Jan 22, 2024
80a8ba8
Add words to cSpell dictionary
Cardosaum Jan 22, 2024
852500c
Merge branch 'main' into cardosaum/add-cargo-risczero-benchmark
Cardosaum Jan 22, 2024
e652c76
chore: sort dependencies
Cardosaum Jan 22, 2024
e1a1bf0
create `risc0-zkvm-methods-core`
Cardosaum Jan 23, 2024
25079ef
Remove unused code and logging initialization
Cardosaum Jan 23, 2024
2856c23
Fix commit_slice argument in multi_test.rs
Cardosaum Jan 23, 2024
945f73d
Merge branch 'main' into cardosaum/add-cargo-risczero-benchmark
Cardosaum Jan 24, 2024
cff8bcd
Revert "feat(cargo-build): Add ability to compile specified binaries."
Cardosaum Jan 24, 2024
153b412
Add xtask to compile benchmark elf
Cardosaum Jan 24, 2024
fb37fcb
Remove unused dependencies in Cargo.toml
Cardosaum Jan 24, 2024
9351187
Add empty line between imports and struct definition
Cardosaum Jan 24, 2024
1ac3c65
cargo fmt
Cardosaum Jan 24, 2024
fb76458
cargo sort
Cardosaum Jan 24, 2024
671a6bf
Add logic to generate `benchmark_elf.rs` during build time
Cardosaum Jan 24, 2024
3f8833c
update cspell
Cardosaum Jan 24, 2024
4572b4e
fix import for risc0-zkvm-methods-core in multiple crates
Cardosaum Jan 24, 2024
09a25f0
cargo sort
Cardosaum Jan 24, 2024
3f38719
Merge branch 'main' into cardosaum/add-cargo-risczero-benchmark
Cardosaum Jan 24, 2024
735a6e6
fix path for risc0-zkvm-methods-core
Cardosaum Jan 25, 2024
20ff703
Fix extra bracket in executor.rs example
Cardosaum Jan 25, 2024
9486165
Update import statement in guest_run.rs
Cardosaum Jan 25, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
3 changes: 2 additions & 1 deletion risc0/cargo-risczero/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,8 @@ reqwest-middleware = "0.2"
reqwest-retry = "0.3"
risc0-build = { workspace = true }
risc0-r0vm = { workspace = true, optional = true }
risc0-zkvm = { workspace = true, optional = true }
risc0-zkvm = { path = "../zkvm", default-features = false, features = ["std", "getrandom"] }
risc0-zkvm-methods = { path = "../zkvm/methods", default-features = false }
Cardosaum marked this conversation as resolved.
Show resolved Hide resolved
serde = { version = "1", features = ["derive"] }
syn = "2.0.38"
tar = "0.4"
Expand Down
1 change: 1 addition & 0 deletions risc0/cargo-risczero/src/bin/cargo-risczero.rs
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ fn main() -> Result<()> {
RisczeroCmd::BuildToolchain(cmd) => cmd.run(),
RisczeroCmd::Install(cmd) => cmd.run(),
RisczeroCmd::New(cmd) => cmd.run(),
RisczeroCmd::Benchmark(cmd) => cmd.run(),
#[cfg(feature = "experimental")]
RisczeroCmd::BuildCrate(build) => build.run(BuildSubcommand::Build),
#[cfg(feature = "experimental")]
Expand Down
111 changes: 111 additions & 0 deletions risc0/cargo-risczero/src/commands/benchmark.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
// Copyright 2024 RISC Zero, Inc.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

use std::time::{Duration, Instant};

use risc0_zkvm_methods::{
bench::{BenchmarkSpec, SpecWithIters},
BENCH_ELF,
};

use risc0_zkvm::{
get_prover_server,
recursion::{join, lift},
ExecutorEnv, ExecutorImpl, ProverOpts, VerifierContext,
};

use anyhow::Result;
use clap::Parser;

/// `cargo risczero benchmark`
#[derive(Parser)]
pub struct BenchmarkCommand {
/// Number of iterations.
#[arg(short, long)]
pub iterations: Option<u64>,

/// Which hash function to use.
#[arg(short = 'f', long, default_value_t = String::from("poseidon"), value_parser = ["poseidon", "sha-256"])]
pub hashfn: String,

/// Specify the segment po2.
#[arg(short, long, default_value_t = 20)]
po2: u32,
}

impl BenchmarkCommand {
/// Execute this command.
pub fn run(&self) -> Result<()> {
// TODO: Handle the case where the user does not specify the number of iterations
let iterations = SpecWithIters(
BenchmarkSpec::SimpleLoop,
self.iterations.unwrap_or(4 * 1024),
);
let env = ExecutorEnv::builder()
.write(&iterations)?
.segment_limit_po2(self.po2)
.build()?;
let mut exec = ExecutorImpl::from_elf(env, BENCH_ELF)?;

// Execute
let (session, exec_duration) = with_duration(|| exec.run())?;

let cycles = session.get_cycles()?;
let segments = session.resolve()?;

let opts = ProverOpts::default();
let ctx = VerifierContext::default();
let prover = get_prover_server(&opts)?;

let mut lifts = vec![];
let mut prove_durations = vec![];
let mut lift_durations = vec![];

// Prove and Lift
for segment in segments.iter() {
let (receipt, receipt_duration) =
with_duration(|| prover.prove_segment(&ctx, segment))?;
prove_durations.push(receipt_duration);

let (lift, lift_duration) = with_duration(|| lift(&receipt))?;
lifts.push(lift);
lift_durations.push(lift_duration);
}

let mut join_durations = vec![];
// Optional Join
if segments.len() > 1 {
let (_final, duration) = with_duration(|| join(&lifts[0], &lifts[1]))?;
join_durations.push(duration);
}

println!("\nSTATS:");
println!("cycles: {}", cycles.1);
println!("segments: {}", segments.len());
println!("exec: {exec_duration:?}");
println!("prove: {prove_durations:?}");
println!("lift: {lift_durations:?}");
println!("prove+lift: {:?}", prove_durations[0] + lift_durations[0]);
println!("join: {join_durations:?}");

Ok(())
}
}

fn with_duration<T, F: FnOnce() -> Result<T>>(f: F) -> Result<(T, Duration)> {
let start = Instant::now();
let result = f()?;
let duration = start.elapsed();
Ok((result, duration))
}
1 change: 1 addition & 0 deletions risc0/cargo-risczero/src/commands/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
// See the License for the specific language governing permissions and
// limitations under the License.

pub mod benchmark;
#[cfg(feature = "experimental")]
pub mod build;
pub mod build_guest;
Expand Down
5 changes: 4 additions & 1 deletion risc0/cargo-risczero/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,8 @@ use clap::{Parser, Subcommand};
#[cfg(feature = "experimental")]
use self::commands::build::BuildCommand;
use self::commands::{
build_guest::BuildGuest, build_toolchain::BuildToolchain, install::Install, new::NewCommand,
benchmark::BenchmarkCommand, build_guest::BuildGuest, build_toolchain::BuildToolchain,
install::Install, new::NewCommand,
};

#[derive(Parser)]
Expand Down Expand Up @@ -58,6 +59,8 @@ pub enum RisczeroCmd {
Install(Install),
/// Creates a new risczero starter project.
New(NewCommand),
/// Prove a zkvm program locally and report performance statistics.
Benchmark(BenchmarkCommand),
/// Build a crate for RISC Zero.
#[cfg(feature = "experimental")]
BuildCrate(BuildCommand),
Expand Down