Test failure report: Perturbative expansion Purification examples, Comparison to MC

[royess]

"Perturbative expansion Purification examples, Comparison to MC" seems to fail randomly. I tried twice, one pass, one failure.

The code I tested is the master branch version 7ff9f39, modified in the QEC module, which should not affect this test.

Test log for "Perturbative expansion Purification examples, Comparison to MC":

Comparison to MC: Test Failed at /home/yuxuan/Documents/Repos/QuantumClifford.jl/test/test_noisycircuits.jl:70
  Expression: compare(mc, pe, false_success_stat)

Stacktrace:
 [1] macro expansion
   @ /snap/julia/95/share/julia/stdlib/v1.10/Test/src/Test.jl:672 [inlined]
 [2] macro expansion
   @ ~/Documents/Repos/QuantumClifford.jl/test/test_noisycircuits.jl:70 [inlined]
 [3] macro expansion
   @ /snap/julia/95/share/julia/stdlib/v1.10/Test/src/Test.jl:1577 [inlined]
 [4] macro expansion
   @ ~/Documents/Repos/QuantumClifford.jl/test/test_noisycircuits.jl:57 [inlined]
 [5] macro expansion
   @ /snap/julia/95/share/julia/stdlib/v1.10/Test/src/Test.jl:1577 [inlined]
 [6] top-level scope
   @ ~/Documents/Repos/QuantumClifford.jl/test/test_noisycircuits.jl:56
Test Summary:                                  | Pass  Fail  Total  Time
noisycircuits                                  |   90     1     91  5.0s
  Noisy Gates                                  |    2            2  1.1s
  Monte Carlo Purification examples            |    9            9  0.4s
  Perturbative expansion Purification examples |   11     1     12  2.7s
    Comparison to MC                           |    8     1      9  1.0s
    Symbolic                                   |    3            3  1.7s
  Measurements                                 |   53           53  0.5s
  Classical Bits                               |   15           15  0.1s
ERROR: LoadError: Some tests did not pass: 90 passed, 1 failed, 0 errored, 0 broken.
in expression starting at /home/yuxuan/Documents/Repos/QuantumClifford.jl/test/runtests.jl:52
ERROR: Package QuantumClifford errored during testing

Environment:

Distributor ID: Ubuntu
Description:    Ubuntu 23.04
Release:        23.04
Codename:       lunar
julia version 1.10.2

[royess]

By the way, does it make sense that we should let testing continue even with some test failures?

[Krastanov]

This is a statistical test, so it is normal for it to fail once in a few thousand runs, but recently it has started failing more often. This might be related to the change to UnbiasedUncorrelatedNoise.