Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid using random numbers for tests #65

Open
king-p3nguin opened this issue Jun 26, 2023 · 3 comments
Open

Avoid using random numbers for tests #65

king-p3nguin opened this issue Jun 26, 2023 · 3 comments
Labels
new feature New feature or request

Comments

@king-p3nguin
Copy link
Contributor

king-p3nguin commented Jun 26, 2023

Is your feature request related to a problem? Please describe.
The use of random numbers in testing can be cited as a reason why transpiler bugs were not found until recently. A simple way to avoid this is to fix the seed value, but the best approach is to increase the number of deterministic test cases that cover various scenarios without relying on random numbers.

Describe the feature you'd like
Create as many test cases as possible to cover various scenarios in tests/test_pattern.py and tests/test_transpiler.py.

Additional context

@king-p3nguin king-p3nguin added the new feature New feature or request label Jun 26, 2023
@shinich1
Copy link
Contributor

@king-p3nguin thank you for the issue report!
Actually we've added a new class of circuits in the tests: see here, random_circuit.py, and tests now run for 10 different realisations of this. Do you think it can be generalised/improved? Please let me know what you think.

@king-p3nguin
Copy link
Contributor Author

king-p3nguin commented Jun 27, 2023

@shinich1
In that case, I think it would be better to test all the circuits that get_rand_circuit can create instead of picking up certain circuits. It will definitely increase the time for testing, but since the pattern optimization is one of the most important features in graphix, it should not be a bad price to pay.

@mgarnier59
Copy link
Contributor

Also, we have starting using the hypothesispackage that is good at generating strategies and finding pathological cases. Sometimes it's a bit too much (like testing for NaN parameter values) but it can be very useful.
We'll take a closer look at the testing strategy and give you feedback!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
new feature New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants