Random failure in test_triangle_inequality seen in ci

[jenshnielsen]

___________________________ test_triangle_inequality ___________________________
[gw0] linux -- Python 3.7.13 /opt/hostedtoolcache/Python/3.7.13/x64/bin/python

    @given(
>       random_coordinates["cylindrical"],
        random_coordinates["spherical"]
    )

/opt/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/site-packages/qcodes/tests/test_field_vector.py:95: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

cylindrical0 = FieldVector(x=1.9984014443252818e-15, y=0.0, z=0.0)
spherical0 = FieldVector(x=0.9975640502598242, y=0.0, z=-0.0[69](https://github.com/QCoDeS/Qcodes/runs/7515516370?check_suite_focus=true#step:10:70)7564[73](https://github.com/QCoDeS/Qcodes/runs/7515516370?check_suite_focus=true#step:10:74)[74](https://github.com/QCoDeS/Qcodes/runs/7515516370?check_suite_focus=true#step:10:75)412533)

    @given(
        random_coordinates["cylindrical"],
        random_coordinates["spherical"]
    )
    @settings(max_examples=10)
    def test_triangle_inequality(cylindrical0, spherical0):
        cylindrical0 = FieldVector(**dict(zip(["rho", "phi", "z"], cylindrical0)))
        spherical0 = FieldVector(**dict(zip(["r", "phi", "theta"], spherical0)))
    
>       assert (cylindrical0 + spherical0).norm() \
               <= (cylindrical0.norm() + spherical0.norm())
E       assert 1.000000000000002 <= (1.9984014443252818e-15 + 0.9999999999999999)
E        +  where 1.000000000000002 = <bound method FieldVector.norm of FieldVector(x=0.99[75](https://github.com/QCoDeS/Qcodes/runs/7515516370?check_suite_focus=true#step:10:76)640502598262, y=0.0, z=-0.06975647374412533)>()
E        +    where <bound method FieldVector.norm of FieldVector(x=0.9975640502598262, y=0.0, z=-0.06975647374412533)> = (FieldVector(x=1.9984014443252[81](https://github.com/QCoDeS/Qcodes/runs/7515516370?check_suite_focus=true#step:10:82)8e-15, y=0.0, z=0.0) + FieldVector(x=0.997564050259[82](https://github.com/QCoDeS/Qcodes/runs/7515516370?check_suite_focus=true#step:10:83)42, y=0.0, z=-0.06975647374412533)).norm
E        +  and   1.99[84](https://github.com/QCoDeS/Qcodes/runs/7515516370?check_suite_focus=true#step:10:85)014443252818e-15 = <bound method FieldVector.norm of FieldVector(x=1.9984014443252818e-15, y=0.0, z=0.0)>()
E        +    where <bound method FieldVector.norm of FieldVector(x=1.9984014443252818e-15, y=0.0, z=0.0)> = FieldVector(x=1.9984014443252818e-15, y=0.0, z=0.0).norm
E        +  and   0.9999999999999999 = <bound method FieldVector.norm of FieldVector(x=0.9975640502598242, y=0.0, z=-0.06975647374412533)>()
E        +    where <bound method FieldVector.norm of FieldVector(x=0.9975640502598242, y=0.0, z=-0.06975647374412533)> = FieldVector(x=0.9975640502598242, y=0.0, z=-0.06975647374412533).norm

/opt/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/site-packages/qcodes/tests/test_field_vector.py:103: AssertionError
---------------------------------- Hypothesis ----------------------------------
Falsifying example: test_triangle_inequality(
    spherical0=(1.0, 0.0, [94](https://github.com/QCoDeS/Qcodes/runs/7515516370?check_suite_focus=true#step:10:95).0),
    cylindrical0=(1.9[98](https://github.com/QCoDeS/Qcodes/runs/7515516370?check_suite_focus=true#step:10:99)4014443252818e-15, 0.0, 0.0),
)

You can reproduce this example by temporarily adding @reproduce_failure('6.53.0', b'AXicY2CAAUYGbCAOiA9DmCxYFTAAACQsASc=') as a decorator on your test case

[fblanchetNaN]

Regarding the values and operations (squares and square roots), it looks like IEEE 754 consequence, no? If so, I don't think it's relevant.
cylindrical0.x/np.spacing(spherical0.x) = 18

[jenshnielsen]

@fblanchetNaN Yes that is likely. Recently a new version of Hypothesis became a lot better at picking float edgecases. The tests likely need to be adjusted to that that into account

[jenshnielsen]

ndrical0 = FieldVector(x=0.0, y=0.0, z=1.13[90](https://github.com/microsoft/Qcodes/actions/runs/7911017030/job/21594535241?pr=5738#step:11:91)888232654144e-13)
spherical0 = FieldVector(x=0.016361631034953292, y=0.0, z=0.93735721420[91](https://github.com/microsoft/Qcodes/actions/runs/7911017030/job/21594535241?pr=5738#step:11:92)168)

    @given(
        random_coordinates["cylindrical"],
        random_coordinates["spherical"]
    )
    @settings(max_examples=10)
    def test_triangle_inequality(cylindrical0, spherical0) -> None:
        cylindrical0 = FieldVector(**dict(zip(["rho", "phi", "z"], cylindrical0)))
        spherical0 = FieldVector(**dict(zip(["r", "phi", "theta"], spherical0)))
    
>       assert (cylindrical0 + spherical0).norm() \
               <= (cylindrical0.norm() + spherical0.norm())
E       assert 0.9375000000001139 <= (1.1390888232654144e-13 + 0.9374999999999999)
E        +  where 0.9375000000001139 = <bound method FieldVector.norm of FieldVector(x=0.0163616310349532[92](https://github.com/microsoft/Qcodes/actions/runs/7911017030/job/21594535241?pr=5738#step:11:93), y=0.0, z=0.[93](https://github.com/microsoft/Qcodes/actions/runs/7911017030/job/21594535241?pr=5738#step:11:94)73572142092307)>()
E        +    where <bound method FieldVector.norm of FieldVector(x=0.016361631034[95](https://github.com/microsoft/Qcodes/actions/runs/7911017030/job/21594535241?pr=5738#step:11:96)3292, y=0.0, z=0.9373572142092307)> = (FieldVector(x=0.0, y=0.0, z=1.1390888232654144e-13) + FieldVector(x=0.016361631034953292, y=0.0, z=0.9373572142091168)).norm
E        +  and   1.1390888232654144e-13 = <bound method FieldVector.norm of FieldVector(x=0.0, y=0.0, z=1.1390888232654144e-13)>()
E        +    where <bound method FieldVector.norm of FieldVector(x=0.0, y=0.0, z=1.1390888232654144e-13)> = FieldVector(x=0.0, y=0.0, z=1.1390888232654144e-13).norm
E        +  and   0.9374[99](https://github.com/microsoft/Qcodes/actions/runs/7911017030/job/21594535241?pr=5738#step:11:100)9999999999 = <bound method FieldVector.norm of FieldVector(x=0.01636163[103](https://github.com/microsoft/Qcodes/actions/runs/7911017030/job/21594535241?pr=5738#step:11:104)4953292, y=0.0, z=0.9373572142091168)>()
E        +    where <bound method FieldVector.norm of FieldVector(x=0.016361631034953292, y=0.0, z=0.9373572142091168)> = FieldVector(x=0.016361631034953292, y=0.0, z=0.9373572142091168).norm
E       Falsifying example: test_triangle_inequality(
E           cylindrical0=(0.0, 0.0, 1.1390888232654144e-13),
E           spherical0=(0.9375, 0.0, 1.0),
E       )
E       
E       You can reproduce this example by temporarily adding @reproduce_failure('6.85.0', b'AXicY2DAAQ6tB5GMIOIARIQdqzpGAFHxAjs=') as a decorator on your test case

another example

https://github.com/microsoft/Qcodes/actions/runs/7911017030/job/21594535241?pr=5738

[jenshnielsen]

Unfortunately this still happens

FAILED tests/test_field_vector.py::test_triangle_inequality - assert 1.0000114441 <= (1.14441e-05 + 0.9999999999999999)
 +  where 1.0000114441 = <bound method FieldVector.norm of FieldVector(x=2.1388503837691124e-06, y=0.0, z=1.0000114440977126)>()
 +    where <bound method FieldVector.norm of FieldVector(x=2.1388503837691124e-06, y=0.0, z=1.0000114440977126)> = (FieldVector(x=0.0, y=0.0, z=1.14441e-05) + FieldVector(x=2.1388503837691124e-06, y=0.0, z=0.9999999999977126)).norm
 +  and   1.14441e-05 = <bound method FieldVector.norm of FieldVector(x=0.0, y=0.0, z=1.14441e-05)>()
 +    where <bound method FieldVector.norm of FieldVector(x=0.0, y=0.0, z=1.14441e-05)> = FieldVector(x=0.0, y=0.0, z=1.14441e-05).norm
 +  and   0.9999999999999999 = <bound method FieldVector.norm of FieldVector(x=2.1388503837691124e-06, y=0.0, z=0.9999999999977126)>()
 +    where <bound method FieldVector.norm of FieldVector(x=2.1388503837691124e-06, y=0.0, z=0.9999999999977126)> = FieldVector(x=2.1388503837691124e-06, y=0.0, z=0.9999999999977126).norm
Falsifying example: test_triangle_inequality(
    cylindrical0=(0.0, 0.0, 1.14441e-05),
    spherical0=(1.0, 0.0, 0.00012[254](https://github.com/microsoft/Qcodes/actions/runs/7956848741/job/21718386518?pr=5747#step:11:255)71),
)

You can reproduce this example by temporarily adding @reproduce_failure('6.85.0', b'AXicY2DAAQ5CKEaECCOqggPnGRkZ//9nAAAwFwRU') as a decorator on your test case
= 1 failed, 2665 passed, 235 skipped, 6 xfailed, [256](https://github.com/microsoft/Qcodes/actions/runs/7956848741/job/21718386518?pr=5747#step:11:257)9 warnings in 401.08s (0:06:41) =
Error: Process completed with exit code 1.