Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

s-dftd3 test refs #370

Open
loriab opened this issue Jul 6, 2022 · 0 comments
Open

s-dftd3 test refs #370

loriab opened this issue Jul 6, 2022 · 0 comments

Comments

@loriab
Copy link
Collaborator

loriab commented Jul 6, 2022

Describe the issue

The s-dftd3 harness #343 originally passed its tests. Later, after an upstream change in physconst, 2 tests (see pytest trace below) started failing in the 8th decimal place. Tests currently loosened to 7 decimal places for 0.24 release. Revisit this to confirm constants are the cause and adjust references.

Additional context

relevant env

# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       2_gnu    conda-forge
cffi                      1.15.0          py310h0fdd8cc_0    conda-forge
dftd3-python              0.6.0           py310h86ebbf9_0    conda-forge
gcp-correction            2.3.1                h1990efc_2    conda-forge
ld_impl_linux-64          2.36.1               hea4e1c9_2    conda-forge
libblas                   3.9.0           15_linux64_openblas    conda-forge
libcblas                  3.9.0           15_linux64_openblas    conda-forge
libffi                    3.4.2                h7f98852_5    conda-forge
libgcc-ng                 12.1.0              h8d9b700_16    conda-forge
libgfortran-ng            12.1.0              h69a702a_16    conda-forge
libgfortran5              12.1.0              hdcd56e2_16    conda-forge
libgomp                   12.1.0              h8d9b700_16    conda-forge
liblapack                 3.9.0           15_linux64_openblas    conda-forge
libopenblas               0.3.20          pthreads_h78a6416_0    conda-forge
libstdcxx-ng              12.1.0              ha89aaad_16    conda-forge
mctc-lib                  0.3.0                hc3ea6d6_0    conda-forge
numpy                     1.23.0          py310h53a5b5f_0    conda-forge
pint                      0.19.2             pyhd8ed1ab_0    conda-forge
pydantic                  1.9.1           py310h5764c6d_0    conda-forge
pytest                    7.1.2           py310hff52083_0    conda-forge
python                    3.10.5          h582c2e5_0_cpython    conda-forge
python_abi                3.10                    2_cp310    conda-forge
qcelemental               0.25.0             pyhd8ed1ab_0    conda-forge
simple-dftd3              0.6.0                h49c083b_0    conda-forge
xtb                       6.5.0                h9d67668_0    conda-forge
xtb-python                20.2            py310h5764c6d_5    conda-forge
=================================== FAILURES ===================================
_______________________ test_dftd3_task_pbe_m02[d3zero] ________________________
inp = {'level_hint': 'd3zero', 'return_result': -0.011922614341086182}
    @using("s-dftd3")
    @pytest.mark.parametrize(
        "inp",
        [
            pytest.param({"return_result": -0.024863196328457682, "level_hint": "d3bj"}),
            pytest.param({"return_result": -0.011922614341086182, "level_hint": "d3zero"}),
            pytest.param({"return_result": -0.054219891333201896, "level_hint": "d3mbj"}),
            pytest.param({"return_result": -0.087100927341320660, "level_hint": "d3mzero"}),
            pytest.param({"return_result": -0.014849063343059512, "level_hint": "d3op"}),
        ],
        ids=["d3bj", "d3zero", "d3mbj", "d3mzero", "d3op"],
    )
    def test_dftd3_task_pbe_m02(inp):
        thr = 1.0e-8
        return_result = inp["return_result"]
        atomic_input = qcel.models.AtomicInput(
            molecule=qcng.get_molecule("mindless-02"),
            model={"method": "pbe"},
            keywords={"level_hint": inp["level_hint"]},
            driver="energy",
        )
        atomic_result = qcng.compute(atomic_input, "s-dftd3")
        assert atomic_result.success
>       assert pytest.approx(atomic_result.return_result, abs=thr) == return_result
E       assert -0.0119225969...9758 ± 1.0e-08 == -0.011922614341086182
E         comparison failed
E         Obtained: -0.011922614341086182
E         Expected: -0.011922596901359758 ± 1.0e-08
qcengine/programs/tests/test_sdftd3.py:64: AssertionError
_______________________ test_dftd3_task_pbe_m02[d3mzero] _______________________
inp = {'level_hint': 'd3mzero', 'return_result': -0.08710092734132066}
    @using("s-dftd3")
    @pytest.mark.parametrize(
        "inp",
        [
            pytest.param({"return_result": -0.024863196328457682, "level_hint": "d3bj"}),
            pytest.param({"return_result": -0.01192261434[1086](https://github.com/MolSSI/QCEngine/runs/7036612683?check_suite_focus=true#step:10:1087)182, "level_hint": "d3zero"}),
            pytest.param({"return_result": -0.054219891333201896, "level_hint": "d3mbj"}),
            pytest.param({"return_result": -0.087100927341320660, "level_hint": "d3mzero"}),
            pytest.param({"return_result": -0.014849063343059512, "level_hint": "d3op"}),
        ],
        ids=["d3bj", "d3zero", "d3mbj", "d3mzero", "d3op"],
    )
    def test_dftd3_task_pbe_m02(inp):
        thr = 1.0e-8
        return_result = inp["return_result"]
        atomic_input = qcel.models.AtomicInput(
            molecule=qcng.get_molecule("mindless-02"),
            model={"method": "pbe"},
            keywords={"level_hint": inp["level_hint"]},
            driver="energy",
        )
        atomic_result = qcng.compute(atomic_input, "s-dftd3")
        assert atomic_result.success
>       assert pytest.approx(atomic_result.return_result, abs=thr) == return_result
E       assert -0.08710099180325154 ± 1.0e-08 == -0.08710092734132066
E         comparison failed
E         Obtained: -0.08710092734132066
E         Expected: -0.08710099180325154 ± 1.0e-08
qcengine/programs/tests/test_sdftd3.py:64: AssertionError
=============================== warnings summary ===============================
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant