Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

1.9.0 system test failures #3165

Closed
4 tasks done
mr-c opened this issue Apr 1, 2024 · 3 comments · Fixed by #3215
Closed
4 tasks done

1.9.0 system test failures #3165

mr-c opened this issue Apr 1, 2024 · 3 comments · Fixed by #3215
Labels
type:Bug Fix

Comments

@mr-c
Copy link
Contributor

mr-c commented Apr 1, 2024

Description

In Debian, we are seeing two test failures for dipy 1.9.0 on 32-bit systems, including x86:
1.

______________________ test_shore_fitting_no_constrain_e0 ______________________

    def test_shore_fitting_no_constrain_e0():
>       asm = ShoreModel(data.gtab, radial_order=data.radial_order,
                         zeta=data.zeta, lambdaN=data.lambdaN,
                         lambdaL=data.lambdaL)
E       AttributeError: '_C' object has no attribute 'radial_order'

dipy/reconst/tests/test_shore.py:70: AttributeError

____________________________ test_check_img_shapes _____________________________

pytestconfig = <_pytest.config.Config object at 0xf7369900>, args = ()
kwargs = {'rng': Generator(PCG64) at 0xE7C5A188}
rng = Generator(PCG64) at 0xE7C5A188, signature = <Signature (rng)>

    def _set_random_number_generator_wrapper(pytestconfig, *args, **kwargs):
        rng = np.random.default_rng(seed_v)
        kwargs['rng'] = rng
        signature = inspect.signature(func)
        if pytestconfig and 'pytestconfig' in signature.parameters.keys():
            output = func(pytestconfig, *args, **kwargs)
        else:
>           output = func(*args, **kwargs)

dipy/testing/decorators.py:95: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
dipy/viz/tests/test_util.py:52: in test_check_img_shapes
    data = 255 * rng.random((198, 233, 189, 14))
numpy/random/_generator.pyx:356: in numpy.random._generator.Generator.random
    ???
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

>   ???
E   numpy.core._exceptions._ArrayMemoryError: Unable to allocate 931. MiB for an array with shape (198, 233, 189, 14) and data type float64

_common.pyx:307: MemoryError

Full i386 (really i686) build log: https://buildd.debian.org/status/fetch.php?pkg=dipy&arch=i386&ver=1.9.0-1&stamp=1711634908&raw=0

Way to reproduce

Here's a Dockerfile:

FROM --platform=linux/i386 docker.io/debian:sid-slim

WORKDIR /build

RUN apt-get update && apt-get install --yes --no-install-recommends devscripts curl debian-keyring
RUN dget https://deb.debian.org/debian/pool/main/d/dipy/dipy_1.9.0-1.dsc
WORKDIR /build/dipy-1.9.0
RUN apt-get -y build-dep .
RUN dpkg-buildpackage -uc -us -b

[If reporting a bug, please include the following important information:]

  • Operating system and version (run python -c "import platform; print(platform.platform())"): Linux-6.1.0-18-amd64-x86_64-with-glibc2.37
  • Python version (run python -c "import sys; print('Python', sys.version)"): Python 3.12.2 (main, Mar 26 2024, 12:39:01) [GCC 13.2.0]
  • dipy version (run python -c "import dipy; print(dipy.__version__)"): 1.9.0
  • dependency version (numpy, scipy, nibabel, h5py, cvxpy, fury)
    • import numpy; print("NumPy", numpy.version): NumPy 1.26.4
    • import scipy; print("SciPy", scipy.version): SciPy 1.11.4
    • import nibabel; print("Nibabel", nibabel.version): Nibabel 5.2.1
    • import h5py; print("H5py", h5py.version): H5py 3.10.0
    • import cvxpy; print("Cvxpy", cvxpy.version): No module named 'cvxpy'
    • import fury; print("fury", fury.version): No module named 'fury
@skoudoro
Copy link
Member

skoudoro commented Apr 1, 2024

Thank you for this report @mr-c.

We need to block time to focus on this. We also need to add a CI to catch this earlier.

@mr-c mr-c changed the title 1.9.0 32-bit system test failures 1.9.0 system test failures May 10, 2024
@mr-c
Copy link
Contributor Author

mr-c commented May 10, 2024

Thank you for the update @skoudoro

We now see the first error (AttributeError: '_C' object has no attribute 'radial_order') for x86-64 systems as well: https://salsa.debian.org/med-team/dipy/-/jobs/5696086

Do you know when you might block time for this?

mr-c added a commit to mr-c/dipy that referenced this issue May 10, 2024
@mr-c
Correct module-level setup
e264f99 
Fixes: dipy#3165

See https://docs.pytest.org/en/7.4.x/how-to/xunit_setup.html
@mr-c mr-c mentioned this issue May 10, 2024
Merged
@AdrianBunk
Copy link

Regarding the 32-bit failure:

E   numpy.core._exceptions._ArrayMemoryError: Unable to allocate 931. MiB for an array with shape (198, 233, 189, 14) and data type float64

4 GiB is the fundamental address space limit on 32-bit architectures, you cannot address more than 2^32 bytes there.

In practice 32-bit architectures have between 2 GiB and 4 GiB address space.

If this is not the only place in the testcase where an array with over 100 million 8 byte variables is being allocated, then that test is too large for 32-bit architectures.

skoudoro pushed a commit that referenced this issue May 14, 2024
@mr-c
Correct module-level setup (#3215)
2a0995b 
Fixes: #3165

See https://docs.pytest.org/en/7.4.x/how-to/xunit_setup.html
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
type:Bug Fix
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

tests: correct module-level setup mr-c/dipy
3 participants
@mr-c @AdrianBunk @skoudoro