Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Unit tests failing #2525

Closed
jdebacker opened this issue Jan 3, 2021 · 8 comments
Closed

Unit tests failing #2525

jdebacker opened this issue Jan 3, 2021 · 8 comments

Comments

@jdebacker
Copy link
Member

As I ran the suite of unit tests on my branch with PR #2519, I find that four tests fail:

FAILED tests/test_cpscsv.py::test_agg - ValueError: CPSCSV AGG RESULTS DIFFER
FAILED tests/test_puf_var_stats.py::test_puf_var_stats - numba.core.errors.Ty...
FAILED tests/test_pufcsv.py::test_agg - ValueError: PUFCSV AGG RESULTS DIFFER...
FAILED tests/test_pufcsv.py::test_puf_availability - AssertionError: assert {...

I didn't think that any changes for that PR should have affected these tests, so I checked out the master branch and the same four tests failed. I had just rebuilt my taxcalc-dev environment and I checked that had the latest puf.csv file.

Differences in the test_cpscsv.py::test_agg and tests/test_pufcsv.py::test_agg are small and mostly related to standard vs itemized deductions in years 2021-2026.

tests/test_pufcsv.py::test_puf_availability fails when looking for the nu05 variable, which I thought was removed in favor of nu06 in PR #2443 (following a change in TaxData).

Does anyone else find these same test failures in the master branch?

cc @MattHJensen @Peter-Metz
@jdebacker jdebacker mentioned this issue Jan 3, 2021
Open
@MattHJensen
Copy link
Contributor

@jdebacker, here are the output from my testing on master this morning. I'm not sure what is going on with the skipped tests from test_compatible_data, but I don't seem to be getting the same errors that you are.

============================= test session starts ==============================
platform darwin -- Python 3.8.5, pytest-6.0.1, py-1.9.0, pluggy-0.13.1
rootdir: /Users/matthewjensen/Projects/Tax-Calculator, configfile: pytest.ini, testpaths: taxcalc
plugins: xdist-2.0.0, forked-1.2.0, pep8-1.0.6
collected 333 items                                                            

taxcalc/tests/test_4package.py ..                                        [  0%]
taxcalc/tests/test_benefits.py .                                         [  0%]
taxcalc/tests/test_calcfunctions.py ...                                  [  1%]
taxcalc/tests/test_calculator.py ....................................... [ 13%]
....                                                                     [ 14%]
taxcalc/tests/test_compare.py ..                                         [ 15%]
taxcalc/tests/test_compatible_data.py .sssssssssssssssssssssssssssssssss [ 25%]
sssssssssss                                                              [ 28%]
taxcalc/tests/test_consumption.py .......                                [ 30%]
taxcalc/tests/test_cpscsv.py ..                                          [ 31%]
taxcalc/tests/test_data.py .                                             [ 31%]
taxcalc/tests/test_decorators.py ................                        [ 36%]
taxcalc/tests/test_growdiff.py .....                                     [ 38%]
taxcalc/tests/test_growfactors.py ....                                   [ 39%]
taxcalc/tests/test_parameters.py ..............................          [ 48%]
taxcalc/tests/test_policy.py ......................................      [ 59%]
taxcalc/tests/test_puf_var_stats.py .                                    [ 60%]
taxcalc/tests/test_pufcsv.py .....                                       [ 61%]
taxcalc/tests/test_records.py ....................                       [ 67%]
taxcalc/tests/test_reforms.py .......................................... [ 80%]
.........................                                                [ 87%]
taxcalc/tests/test_responses.py .                                        [ 87%]
taxcalc/tests/test_taxcalcio.py ..................                       [ 93%]
taxcalc/tests/test_utils.py ......................                       [100%]

================= 289 passed, 44 skipped in 815.00s (0:13:34) ==================

(taxcalc-dev) Matthews-MBP:tax-calculator matthewjensen$ conda list
# packages in environment at /Users/matthewjensen/anaconda3/envs/taxcalc-dev:
#
# Name                    Version                   Build  Channel
alabaster                 0.7.12                   pypi_0    pypi
apipkg                    1.5                        py_0    conda-forge
appnope                   0.1.0                    pypi_0    pypi
argon2-cffi               20.1.0                   pypi_0    pypi
arrow-cpp                 0.15.1           py38hb41def8_6  
astroid                   2.4.2            py38h32f6830_0    conda-forge
async-generator           1.10                     pypi_0    pypi
attrs                     20.1.0             pyh9f0ad1d_0    conda-forge
babel                     2.8.0                    pypi_0    pypi
backcall                  0.2.0                    pypi_0    pypi
beautifulsoup4            4.9.1                    pypi_0    pypi
bleach                    3.1.5                    pypi_0    pypi
bokeh                     2.1.1            py38h32f6830_0    conda-forge
boost-cpp                 1.69.0            h6f8c590_1000    conda-forge
brotli                    1.0.9                hb1e8313_2  
brotlipy                  0.7.0           py38h64e0658_1000    conda-forge
bzip2                     1.0.8                h1de35cc_0  
c-ares                    1.16.1               haf1e3a3_0  
ca-certificates           2020.10.14                    0  
certifi                   2020.6.20          pyhd3eb1b0_3  
cffi                      1.14.1           py38hc4dd44e_0    conda-forge
chardet                   3.0.4           py38h32f6830_1006    conda-forge
click                     7.1.2                    pypi_0    pypi
colorama                  0.4.3                    pypi_0    pypi
coverage                  5.2.1            py38h4d0b108_0    conda-forge
cryptography              3.0              py38h52adbb4_0    conda-forge
decorator                 4.4.2                    pypi_0    pypi
defusedxml                0.6.0                    pypi_0    pypi
docutils                  0.16                     pypi_0    pypi
double-conversion         3.1.5                haf313ee_1  
entrypoints               0.3                      pypi_0    pypi
execnet                   1.7.1                      py_0    conda-forge
freetype                  2.10.2               h8da9a1a_0    conda-forge
fsspec                    0.8.0                      py_0    conda-forge
gflags                    2.2.2                h0a44026_0  
gitdb                     4.0.5                    pypi_0    pypi
gitpython                 3.1.7                    pypi_0    pypi
glog                      0.4.0                h0a44026_0  
grpc-cpp                  1.26.0               h044775b_0  
icu                       58.2                 h0a44026_3  
idna                      2.10               pyh9f0ad1d_0    conda-forge
imagesize                 1.2.0                    pypi_0    pypi
iniconfig                 1.0.1              pyh9f0ad1d_0    conda-forge
ipykernel                 5.3.4                    pypi_0    pypi
ipython                   7.17.0                   pypi_0    pypi
ipython-genutils          0.2.0                    pypi_0    pypi
ipywidgets                7.5.1                    pypi_0    pypi
isort                     4.3.21           py38h32f6830_1    conda-forge
jedi                      0.17.2                   pypi_0    pypi
jinja2                    2.11.2             pyh9f0ad1d_0    conda-forge
jpeg                      9d                   h0b31af3_0    conda-forge
jsonschema                3.2.0                    pypi_0    pypi
jupyter-book              0.7.4                    pypi_0    pypi
jupyter-cache             0.3.0                    pypi_0    pypi
jupyter-client            6.1.6                    pypi_0    pypi
jupyter-core              4.6.3                    pypi_0    pypi
jupyter-sphinx            0.2.4                    pypi_0    pypi
latexcodec                2.0.1                    pypi_0    pypi
lazy-object-proxy         1.4.3            py38h64e0658_2    conda-forge
lcms2                     2.11                 h174193d_0    conda-forge
libblas                   3.8.0               17_openblas    conda-forge
libboost                  1.67.0               hebc422b_4  
libcblas                  3.8.0               17_openblas    conda-forge
libcxx                    10.0.1               h5f48129_0    conda-forge
libevent                  2.1.8                hddc9c9b_1  
libffi                    3.2.1             hb1e8313_1007    conda-forge
libgfortran               4.0.0                         2    conda-forge
libiconv                  1.16                 h1de35cc_0  
liblapack                 3.8.0               17_openblas    conda-forge
libllvm10                 10.0.1               h009f743_1    conda-forge
libopenblas               0.3.10          openmp_h63d9170_4    conda-forge
libpng                    1.6.37               hb0a8c7a_2    conda-forge
libprotobuf               3.11.2               hd9629dc_0  
libsass                   0.20.0                   pypi_0    pypi
libtiff                   4.1.0                hcb84e12_0  
libwebp-base              1.1.0                h0b31af3_3    conda-forge
llvm-openmp               10.0.1               h28b9765_0    conda-forge
llvmlite                  0.34.0           py38h3707e27_0    conda-forge
lz4-c                     1.8.1.2              h1de35cc_0  
markdown-it-py            0.4.9                    pypi_0    pypi
markupsafe                1.1.1            py38h64e0658_1    conda-forge
marshmallow               3.7.1                      py_0    conda-forge
mccabe                    0.6.1                      py_1    conda-forge
mistune                   0.8.4                    pypi_0    pypi
more-itertools            8.4.0                      py_0    conda-forge
myst-nb                   0.8.5                    pypi_0    pypi
myst-parser               0.9.1                    pypi_0    pypi
nbclient                  0.4.1                    pypi_0    pypi
nbconvert                 5.6.1                    pypi_0    pypi
nbdime                    2.0.0                    pypi_0    pypi
nbformat                  5.0.7                    pypi_0    pypi
ncurses                   6.2                  hb1e8313_1    conda-forge
nest-asyncio              1.4.0                    pypi_0    pypi
notebook                  6.1.3                    pypi_0    pypi
numba                     0.51.0           py38h6be0db6_0    conda-forge
numpy                     1.19.1           py38h8ccc501_2    conda-forge
olefile                   0.46                       py_0    conda-forge
openssl                   1.1.1h               haf1e3a3_0  
oset                      0.1.3                    pypi_0    pypi
packaging                 20.4               pyh9f0ad1d_0    conda-forge
pandas                    1.1.0            py38h11c0d25_0    conda-forge
pandocfilters             1.4.2                    pypi_0    pypi
paramtools                0.15.0             pyh9f0ad1d_0    conda-forge
parso                     0.7.1                    pypi_0    pypi
pep8                      1.7.1                      py_0    conda-forge
pexpect                   4.8.0                    pypi_0    pypi
pickleshare               0.7.5                    pypi_0    pypi
pillow                    7.2.0            py38h83dc5e5_1    conda-forge
pip                       20.2.2                     py_0    conda-forge
pluggy                    0.13.1           py38h32f6830_2    conda-forge
prometheus-client         0.8.0                    pypi_0    pypi
prompt-toolkit            3.0.6                    pypi_0    pypi
psutil                    5.7.2            py38h4d0b108_0    conda-forge
ptyprocess                0.6.0                    pypi_0    pypi
py                        1.9.0              pyh9f0ad1d_0    conda-forge
pyarrow                   0.15.1           py38h6c726b0_0  
pybtex                    0.22.2                   pypi_0    pypi
pybtex-docutils           0.2.2                    pypi_0    pypi
pycodestyle               2.6.0              pyh9f0ad1d_0    conda-forge
pycparser                 2.20               pyh9f0ad1d_2    conda-forge
pydata-sphinx-theme       0.3.1                    pypi_0    pypi
pygments                  2.6.1                    pypi_0    pypi
pylint                    2.5.3            py38h32f6830_0    conda-forge
pyopenssl                 19.1.0                     py_1    conda-forge
pyparsing                 2.4.7              pyh9f0ad1d_0    conda-forge
pyrsistent                0.16.0                   pypi_0    pypi
pysocks                   1.7.1            py38h32f6830_1    conda-forge
pytest                    6.0.1            py38h32f6830_0    conda-forge
pytest-cache              1.0                        py_2    conda-forge
pytest-forked             1.2.0              pyh9f0ad1d_0    conda-forge
pytest-pep8               1.0.6                      py_2    conda-forge
pytest-xdist              2.0.0                      py_0    conda-forge
python                    3.8.5           hfc71d35_4_cpython    conda-forge
python-dateutil           2.8.1                      py_0    conda-forge
python_abi                3.8                      1_cp38    conda-forge
pytz                      2020.1             pyh9f0ad1d_0    conda-forge
pyyaml                    5.3.1            py38h64e0658_0    conda-forge
pyzmq                     19.0.2                   pypi_0    pypi
re2                       2019.08.01           h0a44026_0  
readline                  8.0                  h0678c8f_2    conda-forge
requests                  2.24.0             pyh9f0ad1d_0    conda-forge
send2trash                1.5.0                    pypi_0    pypi
setuptools                49.6.0           py38h32f6830_0    conda-forge
six                       1.15.0             pyh9f0ad1d_0    conda-forge
smmap                     3.0.4                    pypi_0    pypi
snappy                    1.1.8                hb1e8313_0  
snowballstemmer           2.0.0                    pypi_0    pypi
soupsieve                 2.0.1                    pypi_0    pypi
sphinx                    2.4.4                    pypi_0    pypi
sphinx-book-theme         0.0.35                   pypi_0    pypi
sphinx-copybutton         0.3.0                    pypi_0    pypi
sphinx-thebe              0.0.7                    pypi_0    pypi
sphinx-togglebutton       0.2.2                    pypi_0    pypi
sphinxcontrib-applehelp   1.0.2                    pypi_0    pypi
sphinxcontrib-bibtex      1.0.0                    pypi_0    pypi
sphinxcontrib-devhelp     1.0.2                    pypi_0    pypi
sphinxcontrib-htmlhelp    1.0.3                    pypi_0    pypi
sphinxcontrib-jsmath      1.0.1                    pypi_0    pypi
sphinxcontrib-qthelp      1.0.3                    pypi_0    pypi
sphinxcontrib-serializinghtml 1.1.4                    pypi_0    pypi
sqlalchemy                1.3.19                   pypi_0    pypi
sqlite                    3.33.0               h960bd1c_0    conda-forge
terminado                 0.8.3                    pypi_0    pypi
testpath                  0.4.4                    pypi_0    pypi
thrift-cpp                0.11.0               hd79cdb6_3  
tk                        8.6.10               hb0a8c7a_0    conda-forge
toml                      0.10.1             pyh9f0ad1d_0    conda-forge
tornado                   6.0.4            py38h64e0658_1    conda-forge
traitlets                 4.3.3                    pypi_0    pypi
typing_extensions         3.7.4.2                    py_0    conda-forge
uriparser                 0.9.3                h0a44026_1  
urllib3                   1.25.10                    py_0    conda-forge
wcwidth                   0.2.5                    pypi_0    pypi
webencodings              0.5.1                    pypi_0    pypi
wheel                     0.35.1             pyh9f0ad1d_0    conda-forge
widgetsnbextension        3.5.1                    pypi_0    pypi
wrapt                     1.11.2           py38h64e0658_0    conda-forge
xz                        5.2.5                haf1e3a3_1    conda-forge
yaml                      0.2.5                haf1e3a3_0    conda-forge
zlib                      1.2.11                     1007    conda-forge
zstd                      1.3.7                h5bba6e5_0

@Peter-Metz
Copy link
Contributor

Initially, I didn't run into any failures, but after rebuilding my taxcalc-dev environment, I ran into the same failures as @jdebacker, leading me to believe that there was an issue with dependency versions. Turns out pandas is the culprit. Downgrading from 1.2.0 to 1.1.0 does the trick.

Looking at the pandas release notes, it seems like the difference is coming from a change in default floating precision for read_csv (the parser is now more accurate). Seems like we should update the expected outputs in the taxcalc tests folder and pin pandas >=1.2.0 in the taxcalc environment.

cc @MattHJensen

@jdebacker
Copy link
Member Author

@MattHJensen Thanks for running the tests. Can you update your environment? I have Python 3.8.6 and later versions of Numba, Pandas, ParamTools (and other packages - but those seem most relevant).

My conda list:

# Name                    Version                   Build  Channel
alabaster                 0.7.12                   pypi_0    pypi
apipkg                    1.5                        py_0    conda-forge
appnope                   0.1.2                    pypi_0    pypi
argon2-cffi               20.1.0                   pypi_0    pypi
astroid                   2.4.2            py38h32f6830_1    conda-forge
async-generator           1.10                     pypi_0    pypi
attrs                     20.3.0             pyhd3deb0d_0    conda-forge
babel                     2.9.0                    pypi_0    pypi
backcall                  0.2.0                    pypi_0    pypi
beautifulsoup4            4.9.3                    pypi_0    pypi
bleach                    3.2.1                    pypi_0    pypi
bokeh                     2.2.3            py38h50d1736_0    conda-forge
brotlipy                  0.7.0           py38h94c058a_1001    conda-forge
ca-certificates           2020.12.5            h033912b_0    conda-forge
certifi                   2020.12.5        py38h50d1736_0    conda-forge
cffi                      1.14.4           py38h979bc6b_1    conda-forge
chardet                   4.0.0            py38h50d1736_0    conda-forge
click                     7.1.2                    pypi_0    pypi
colorama                  0.4.4                    pypi_0    pypi
coverage                  5.3.1            py38h5406a74_0    conda-forge
cryptography              3.3.1            py38h6b4ec92_0    conda-forge
decorator                 4.4.2                    pypi_0    pypi
defusedxml                0.6.0                    pypi_0    pypi
docutils                  0.16                     pypi_0    pypi
entrypoints               0.3                      pypi_0    pypi
execnet                   1.7.1                      py_0    conda-forge
freetype                  2.10.4               h3f75d11_0    conda-forge
fsspec                    0.8.5              pyhd8ed1ab_0    conda-forge
gitdb                     4.0.5                    pypi_0    pypi
gitpython                 3.1.11                   pypi_0    pypi
idna                      2.10               pyh9f0ad1d_0    conda-forge
imagesize                 1.2.0                    pypi_0    pypi
importlib-metadata        3.3.0                    pypi_0    pypi
iniconfig                 1.1.1              pyh9f0ad1d_0    conda-forge
ipykernel                 5.4.2                    pypi_0    pypi
ipython                   7.19.0                   pypi_0    pypi
ipython-genutils          0.2.0                    pypi_0    pypi
ipywidgets                7.6.2                    pypi_0    pypi
isort                     5.7.0              pyhd8ed1ab_0    conda-forge
jedi                      0.18.0                   pypi_0    pypi
jinja2                    2.11.2             pyh9f0ad1d_0    conda-forge
jpeg                      9d                   hbcb3906_0    conda-forge
jsonschema                3.2.0                    pypi_0    pypi
jupyter-book              0.9.1                    pypi_0    pypi
jupyter-cache             0.4.1                    pypi_0    pypi
jupyter-client            6.1.7                    pypi_0    pypi
jupyter-core              4.7.0                    pypi_0    pypi
jupyter-sphinx            0.3.2                    pypi_0    pypi
jupyterlab-widgets        1.0.0                    pypi_0    pypi
latexcodec                2.0.1                    pypi_0    pypi
lazy-object-proxy         1.4.3            py38h4d0b108_2    conda-forge
lcms2                     2.11                 h11f7e16_1    conda-forge
libblas                   3.9.0                6_openblas    conda-forge
libcblas                  3.9.0                6_openblas    conda-forge
libcxx                    11.0.0               h4c3b8ed_1    conda-forge
libffi                    3.3                  h046ec9c_2    conda-forge
libgfortran               5.0.0               h6c81a4c_15    conda-forge
libgfortran5              9.3.0               h6c81a4c_15    conda-forge
liblapack                 3.9.0                6_openblas    conda-forge
libllvm10                 10.0.1               h009f743_3    conda-forge
libopenblas               0.3.12          openmp_h54245bb_1    conda-forge
libpng                    1.6.37               h7cec526_2    conda-forge
libtiff                   4.2.0                h355d032_0    conda-forge
libwebp-base              1.1.0                hbcb3906_3    conda-forge
llvm-openmp               11.0.0               h73239a0_1    conda-forge
llvmlite                  0.35.0           py38hc583f44_0    conda-forge
lz4-c                     1.9.3                h046ec9c_0    conda-forge
markdown-it-py            0.5.8                    pypi_0    pypi
markupsafe                1.1.1            py38h94c058a_2    conda-forge
marshmallow               3.10.0             pyhd8ed1ab_0    conda-forge
mccabe                    0.6.1                      py_1    conda-forge
mistune                   0.8.4                    pypi_0    pypi
more-itertools            8.6.0              pyhd8ed1ab_0    conda-forge
myst-nb                   0.10.1                   pypi_0    pypi
myst-parser               0.12.10                  pypi_0    pypi
nbclient                  0.5.1                    pypi_0    pypi
nbconvert                 5.6.1                    pypi_0    pypi
nbdime                    2.1.0                    pypi_0    pypi
nbformat                  5.0.8                    pypi_0    pypi
ncurses                   6.2                  h2e338ed_4    conda-forge
nest-asyncio              1.4.3                    pypi_0    pypi
nested-lookup             0.2.21                   pypi_0    pypi
notebook                  6.1.6                    pypi_0    pypi
numba                     0.52.0           py38he9f00de_0    conda-forge
numpy                     1.19.4           py38h6ced74f_2    conda-forge
olefile                   0.46               pyh9f0ad1d_1    conda-forge
openssl                   1.1.1i               h35c211d_0    conda-forge
oset                      0.1.3                    pypi_0    pypi
packaging                 20.8               pyhd3deb0d_0    conda-forge
pandas                    1.2.0            py38he9f00de_0    conda-forge
pandocfilters             1.4.3                    pypi_0    pypi
paramtools                0.16.0             pyhd3deb0d_0    conda-forge
parso                     0.8.1                    pypi_0    pypi
pep8                      1.7.1                      py_0    conda-forge
pexpect                   4.8.0                    pypi_0    pypi
pickleshare               0.7.5                    pypi_0    pypi
pillow                    8.0.1            py38h565d989_0    conda-forge
pip                       20.3.3             pyhd8ed1ab_0    conda-forge
pluggy                    0.13.1           py38h5347e94_3    conda-forge
prometheus-client         0.9.0                    pypi_0    pypi
prompt-toolkit            3.0.8                    pypi_0    pypi
ptyprocess                0.7.0                    pypi_0    pypi
py                        1.10.0             pyhd3deb0d_0    conda-forge
pybtex                    0.23.0                   pypi_0    pypi
pybtex-docutils           0.2.2                    pypi_0    pypi
pycodestyle               2.6.0              pyh9f0ad1d_0    conda-forge
pycparser                 2.20               pyh9f0ad1d_2    conda-forge
pydata-sphinx-theme       0.4.1                    pypi_0    pypi
pygments                  2.7.3                    pypi_0    pypi
pylint                    2.6.0            py38h32f6830_1    conda-forge
pyopenssl                 20.0.1             pyhd8ed1ab_0    conda-forge
pyparsing                 2.4.7              pyh9f0ad1d_0    conda-forge
pyrsistent                0.17.3                   pypi_0    pypi
pysocks                   1.7.1            py38h5347e94_2    conda-forge
pytest                    6.2.1            py38h50d1736_0    conda-forge
pytest-cache              1.0                        py_2    conda-forge
pytest-forked             1.2.0              pyh9f0ad1d_0    conda-forge
pytest-pep8               1.0.6                      py_2    conda-forge
pytest-xdist              2.2.0              pyhd8ed1ab_0    conda-forge
python                    3.8.6           h624753d_4_cpython    conda-forge
python-dateutil           2.8.1                      py_0    conda-forge
python_abi                3.8                      1_cp38    conda-forge
pytz                      2020.5             pyhd8ed1ab_0    conda-forge
pyyaml                    5.3.1            py38h94c058a_1    conda-forge
pyzmq                     20.0.0                   pypi_0    pypi
readline                  8.0                  h0678c8f_2    conda-forge
requests                  2.25.1             pyhd3deb0d_0    conda-forge
send2trash                1.5.0                    pypi_0    pypi
setuptools                49.6.0           py38h5347e94_2    conda-forge
six                       1.15.0             pyh9f0ad1d_0    conda-forge
smmap                     3.0.4                    pypi_0    pypi
snowballstemmer           2.0.0                    pypi_0    pypi
sortedcontainers          2.3.0              pyhd8ed1ab_0    conda-forge
soupsieve                 2.1                      pypi_0    pypi
sphinx                    3.4.1                    pypi_0    pypi
sphinx-book-theme         0.0.39                   pypi_0    pypi
sphinx-comments           0.0.3                    pypi_0    pypi
sphinx-copybutton         0.3.1                    pypi_0    pypi
sphinx-panels             0.5.2                    pypi_0    pypi
sphinx-thebe              0.0.8                    pypi_0    pypi
sphinx-togglebutton       0.2.3                    pypi_0    pypi
sphinxcontrib-applehelp   1.0.2                    pypi_0    pypi
sphinxcontrib-bibtex      1.0.0                    pypi_0    pypi
sphinxcontrib-devhelp     1.0.2                    pypi_0    pypi
sphinxcontrib-htmlhelp    1.0.3                    pypi_0    pypi
sphinxcontrib-jsmath      1.0.1                    pypi_0    pypi
sphinxcontrib-qthelp      1.0.3                    pypi_0    pypi
sphinxcontrib-serializinghtml 1.1.4                    pypi_0    pypi
sqlalchemy                1.3.22                   pypi_0    pypi
sqlite                    3.34.0               h17101e1_0    conda-forge
terminado                 0.9.1                    pypi_0    pypi
testpath                  0.4.4                    pypi_0    pypi
tk                        8.6.10               h0419947_1    conda-forge
toml                      0.10.2             pyhd8ed1ab_0    conda-forge
tornado                   6.1              py38h7e3306e_0    conda-forge
traitlets                 5.0.5                    pypi_0    pypi
typing_extensions         3.7.4.3                    py_0    conda-forge
urllib3                   1.26.2             pyhd8ed1ab_0    conda-forge
wcwidth                   0.2.5                    pypi_0    pypi
webencodings              0.5.1                    pypi_0    pypi
wheel                     0.36.2             pyhd3deb0d_0    conda-forge
widgetsnbextension        3.5.1                    pypi_0    pypi
wrapt                     1.11.2           py38h4d0b108_1    conda-forge
xz                        5.2.5                haf1e3a3_1    conda-forge
yaml                      0.2.5                haf1e3a3_0    conda-forge
zipp                      3.4.0                    pypi_0    pypi
zlib                      1.2.11            h7795811_1010    conda-forge
zstd                      1.4.8                hf387650_1    conda-forge

@jdebacker
Copy link
Member Author

@Peter-Metz Thanks for digging into this. I'll open a PR with those changes you suggest now.

@jdebacker
Copy link
Member Author

@Peter-Metz I'm still getting a test failure with one test, test_puf_var_stats.py.

Here's the output from that test:

numba.core.errors.TypingError: Failed in nopython mode pipeline (step: nopython frontend)
E           non-precise type pyobject

This is with Numba v. 0.52.0. I tried with Numba 0.51.0 (as in @MattHJensen's env) and still found this test to fail.

=================================== FAILURES ===================================
______________________________ test_puf_var_stats ______________________________

tests_path = '/Users/jason.debacker/repos/tax-calculator/taxcalc/tests'
puf_fullsample =         fips  nu06  nu13  nu18  ...  agi_bin  g20500  pencon_p  pencon_s
0         51     0     0     0  ...        1 ...       0         0
248590    15     0     0     0  ...        1       0         0         0

[248591 rows x 89 columns]

    @pytest.mark.requires_pufcsv
    def test_puf_var_stats(tests_path, puf_fullsample):
        """
        Main logic of test.
        """
        # create a baseline Policy object containing 2017_law.json parameters
        pre_tcja_jrf = os.path.join(tests_path, '..', 'reforms', '2017_law.json')
        pre_tcja = Policy.read_json_reform(pre_tcja_jrf)
        baseline_policy = Policy()
        baseline_policy.implement_reform(pre_tcja)
        # create a Calculator object using baseline_policy and full puf.csv sample
        rec = Records(data=puf_fullsample)
        calc = Calculator(policy=baseline_policy, records=rec, verbose=False)
        # create base tables
        table_mean = create_base_table(tests_path)
        table_corr = copy.deepcopy(table_mean)
        del table_corr['description']
        # add statistics to tables
        year_headers = ['description']
        for year in range(Policy.JSON_START_YEAR, Policy.LAST_BUDGET_YEAR + 1):
            assert year == calc.current_year
            year_headers.append(str(year))
>           calc.calc_all()

tests/test_puf_var_stats.py:176:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
calculator.py:170: in calc_all
    UBI(self.__policy, self.__records)
decorators.py:320: in wrapper
    ans = high_level_fn(*args, **kwargs)
<string>:12: in hl_func
    ???
../../../anaconda3/envs/taxcalc-dev/lib/python3.8/site-packages/numba/core/dispatcher.py:414: in _compile_for_args
    error_rewrite(e, 'typing')
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

e = TypingError('Failed in nopython mode pipeline (step: nopython frontend)\n\x1b[1m\x1b[1mnon-precise type pyobject\x1b[0...lowing argument(s):\n- argument 9: \x1b[1mCannot determine Numba type of <class \'collections.OrderedDict\'>\x1b[0m\n')
issue_type = 'typing'

    def error_rewrite(e, issue_type):
        """
        Rewrite and raise Exception `e` with help supplied based on the
        specified issue_type.
        """
        if config.SHOW_HELP:
            help_msg = errors.error_extras[issue_type]
            e.patch_message('\n'.join((str(e).rstrip(), help_msg)))
        if config.FULL_TRACEBACKS:
            raise e
        else:
>           raise e.with_traceback(None)
E           numba.core.errors.TypingError: Failed in nopython mode pipeline (step: nopython frontend)
E           non-precise type pyobject
E           During: typing of argument at <string> (2)
E
E           File "<string>", line 2:
E           <source missing, REPL/exec in use?>
E
E           This error may have been caused by the following argument(s):
E           - argument 9: Cannot determine Numba type of <class 'collections.OrderedDict'>

../../../anaconda3/envs/taxcalc-dev/lib/python3.8/site-packages/numba/core/dispatcher.py:357: TypingError
=============================== warnings summary ===============================
taxcalc/tests/test_calculator.py: 4 warnings
taxcalc/tests/test_policy.py: 17 warnings
taxcalc/tests/test_puf_var_stats.py: 1 warning
taxcalc/tests/test_reforms.py: 64 warnings
taxcalc/tests/test_taxcalcio.py: 6 warnings
  /Users/jason.debacker/anaconda3/envs/taxcalc-dev/lib/python3.8/site-packages/paramtools/select.py:20: UserWarning: The select module is deprecated. Use Values instead.
    warnings.warn("The select module is deprecated. Use Values instead.")

-- Docs: https://docs.pytest.org/en/stable/warnings.html
=========================== short test summary info ============================
FAILED tests/test_puf_var_stats.py::test_puf_var_stats - numba.core.errors.Ty...
====== 1 failed, 287 passed, 44 skipped, 92 warnings in 906.03s (0:15:06) ======

@jdebacker jdebacker mentioned this issue Jan 4, 2021
Merged
@Peter-Metz
Copy link
Contributor

Ah, this one appears to be coming from paramtools. The test passes with paramtools 0.15.0 but not 0.16.0. @hdoupe do you have any guidance?

@hdoupe
Copy link
Collaborator

hdoupe commented Jan 4, 2021

Hmmm, let me take a closer look @Peter-Metz

@hdoupe
Copy link
Collaborator

hdoupe commented Jan 5, 2021

@jdebacker @Peter-Metz I think I fixed the problem here: jdebacker#1

It's more of a temporary fix, but it should get the unit tests passing again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@MattHJensen @hdoupe @jdebacker @Peter-Metz