Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add framework for benchmarks #1502

Open
wants to merge 8 commits into
base: develop
Choose a base branch
from
Open

Add framework for benchmarks #1502

wants to merge 8 commits into from

Conversation

fsimonis
Copy link
Member

@fsimonis fsimonis commented Dec 2, 2022

Main changes of this PR

This PR adds a framework for implementing benchmarks using the popular benchmark.
As this introduces another unnecessary dependency, the generation of the benchmark suite is disabled by default.

cmake -DPRECICE_ENABLE_BENCHMARKS=ON .. enables the generation of the benchmark executable benchprecice.

Add your benchmark as a .cpp file in the benchmarks/ directory and run tools/building/updateSourceFiles.py.

Motivation and additional information

This is a first step towards performance regression testing.

There are currently no benchmarks.

Author's checklist

  • I used the pre-commit hook to prevent dirty commits and used pre-commit run --all to format old commits.
  • I added a changelog file with make changelog if there are user-observable changes since the last release.
  • I added a test to cover the proposed changes in our test suite.
  • For breaking changes: I documented the changes in the appropriate porting guide.
  • I sticked to C++17 features.
  • I sticked to CMake version 3.16.3.
  • I squashed / am about to squash all commits that should be seen as one.

Reviewers' checklist

  • Does the changelog entry make sense? Is it formatted correctly?
  • Do you understand the code changes?

@fsimonis fsimonis added this to the Version 3.0.0 milestone Dec 2, 2022
@fsimonis fsimonis added enhancement A new feature, a new functionality of preCICE (from user perspective) maintainability Working on this will make our lives easier in the long run as preCICE gets easier to maintain. labels Dec 2, 2022
@fsimonis
Copy link
Member Author

fsimonis commented Dec 2, 2022

Open question: should we build these in the CI pipeline as well?

Copy link
Member

@davidscn davidscn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For now, I only benchmarked 'non-coupled' scenarios. I'm not sure if we can benchmark coupled API functions at all.

CMakeLists.txt Outdated
if(PRECICE_ENABLE_BENCHMARKS)
find_package(benchmark REQUIRED)

add_executable(benchprecice)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In accordance to other executables around preCICE, I would call this precice-bench. I would even consider merging it optionally with the precice-tools

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. Renaming testprecice would also make sense.

@davidscn
Copy link
Member

davidscn commented Dec 5, 2022

Open question: should we build these in the CI pipeline as well?

Experience states that it will become outdated otherwise.

@fsimonis
Copy link
Member Author

fsimonis commented Dec 6, 2022

For now, I only benchmarked 'non-coupled' scenarios. I'm not sure if we can benchmark coupled API functions at all.

Right. We will be able to benchmark internals, such as Mappings, Meshes, Exporters as well as everything SolverInterface-related until the call to initialize.

Experience states that it will become outdated otherwise.

Sure. This simply means that we have to add this dependency to all ci images.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement A new feature, a new functionality of preCICE (from user perspective) maintainability Working on this will make our lives easier in the long run as preCICE gets easier to maintain.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants