Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cargo bolero repro command? #114

Open
Ekleog opened this issue Jan 22, 2023 · 2 comments
Open

cargo bolero repro command? #114

Ekleog opened this issue Jan 22, 2023 · 2 comments
Labels
[C] Feature / Enhancement A new feature request or enhancement to an existing feature. [E] User Experience An UX enhancement for an existing feature. Including deprecation of an existing one.

Comments

@Ekleog
Copy link
Contributor

Ekleog commented Jan 22, 2023

I can't find a way to repro a single fuzzer crash from the cargo bolero command line.

Am I missing something, or should we add such a way, eg. cargo bolero repro [fuzzer] [crash-artifact]?

Currently, the best I found is to use cargo test: it compiles much faster and starts by testing the fuzzer crashes, but even it actually tests all the fuzzer crashes and corpus, and not just the one I'd like to test; which slows down the edit-compile-test cycle

Ekleog added a commit to Ekleog/bolero that referenced this issue Jan 22, 2023
This makes cargo test likely to crash faster than the other way around,
which in turn leads to faster iteration rate

This is a small improvement compared to camshaft#114, but always worth it even
if camshaft#114 were to land
Ekleog added a commit to Ekleog/bolero that referenced this issue Jan 22, 2023
This makes cargo test likely to crash faster than the other way around,
which in turn leads to faster iteration rate

This is a small improvement compared to camshaft#114, but always worth it even
if camshaft#114 were to land
@camshaft
Copy link
Owner

I usually just use cargo test [testname] for this workflow. This is most likely the fastest way possible since we'd have to run cargo test --no-run in the cargo bolero command anyway.

It does mean that if you get a sanitizer failure, it likely won't fail inside cargo test. For that, you can use cargo bolero test [testname] --release false and compilation should be a bit quicker. The fuzzers do generally run the crashes first as well.

@Ekleog
Copy link
Contributor Author

Ekleog commented Jan 28, 2023

Hmm so what you're saying makes sense for a development point of view when someone already knows of how cargo bolero works. (well, and now that #115 landed, as before that libfuzzer fuzzers didn't actually start by running the crashes, and with a 30k corpus at 100exec/s it's noticeable 😅 )

However, if someone doesn't know of how bolero works, having a command to "do whatever but rerun the crash in this file" would make sense I think. For instance, if this person downloaded the crash file from a fuzzer infra like oss-fuzz (though this is pending on #98 for now, but let's assume it's already possible), and doesn't even have the corpus/ and crashes/ folders locally, being able to cargo bolero repro -p [package] [test] [artifact] and that'd run like cargo bolero test -p [package] [test] --release false --engine-args [artifact] (or similar for non-libfuzzer harnesses) would make sense I think.

Would you be open to a PR adding such a subcommand? :)

@adpaco-aws adpaco-aws added [C] Feature / Enhancement A new feature request or enhancement to an existing feature. [E] User Experience An UX enhancement for an existing feature. Including deprecation of an existing one. labels Mar 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
[C] Feature / Enhancement A new feature request or enhancement to an existing feature. [E] User Experience An UX enhancement for an existing feature. Including deprecation of an existing one.
Projects
None yet
Development

No branches or pull requests

3 participants