-
-
Notifications
You must be signed in to change notification settings - Fork 169
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add framework for benchmarks #1502
base: develop
Are you sure you want to change the base?
Conversation
Open question: should we build these in the CI pipeline as well? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For now, I only benchmarked 'non-coupled' scenarios. I'm not sure if we can benchmark coupled API functions at all.
CMakeLists.txt
Outdated
if(PRECICE_ENABLE_BENCHMARKS) | ||
find_package(benchmark REQUIRED) | ||
|
||
add_executable(benchprecice) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In accordance to other executables around preCICE, I would call this precice-bench
. I would even consider merging it optionally with the precice-tools
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good point. Renaming testprecice
would also make sense.
Experience states that it will become outdated otherwise. |
Right. We will be able to benchmark internals, such as Mappings, Meshes, Exporters as well as everything SolverInterface-related until the call to initialize.
Sure. This simply means that we have to add this dependency to all ci images. |
5dc3256
to
f9d4fa7
Compare
Main changes of this PR
This PR adds a framework for implementing benchmarks using the popular benchmark.
As this introduces another unnecessary dependency, the generation of the benchmark suite is disabled by default.
cmake -DPRECICE_ENABLE_BENCHMARKS=ON ..
enables the generation of the benchmark executablebenchprecice
.Add your benchmark as a
.cpp
file in thebenchmarks/
directory and runtools/building/updateSourceFiles.py
.Motivation and additional information
This is a first step towards performance regression testing.
There are currently no benchmarks.
Author's checklist
pre-commit
hook to prevent dirty commits and usedpre-commit run --all
to format old commits.make changelog
if there are user-observable changes since the last release.Reviewers' checklist