Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change group setting in benchmarks #8

Open
xspronken opened this issue Aug 9, 2022 · 6 comments
Open

Change group setting in benchmarks #8

xspronken opened this issue Aug 9, 2022 · 6 comments
Assignees

Comments

@xspronken
Copy link
Collaborator

As @hodgestar mentioned in #6 the contents of group should probably be changed.
For the benchmarks in the existing PR i would suggest somehting like Qobj-operation and QobjEvo-operation or maybe static and timedependent instead of Qobj and QobjEvo.

@hodgestar
Copy link
Contributor

The information currently recorded is:

  "group": "Add-2-dense-numpy",
  "name": "test_add[2-dense-numpy]",
  "fullname": "test_linear_algebra.py::test_add[2-dense-numpy]",
  "params": {
    "size": 2,
    "density": "dense",
    "dtype": "numpy"
  }

@hodgestar
Copy link
Contributor

For each group me might want to add a parameter or extra_param named "implementation" that specifies the implementation used, e.g. "v5-dense", "julia", "v5-sparse", "v4-sparse", ... . I think the name of the test sort of gives this information, but not explicitly, and combines it with the group information. Have a variable that clearly names the implementation would be useful.

@hodgestar
Copy link
Contributor

pytest-benchmark provides an existing mechanism for setting these things that looks quite nice -- https://pytest-benchmark.readthedocs.io/en/latest/usage.html#markers

@hodgestar
Copy link
Contributor

@xspronken
Copy link
Collaborator Author

The groups are currently being changed in #22.

@hodgestar
Copy link
Contributor

I'd like to keep this open or create new issues for the ideas of adding a decorator (mostly stylistic) and an implementation extra_info (which would be useful for organizing results by implementation).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants