New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Patch for len of float assertion error in nastran #1437

Open

RemDelaporteMathurin wants to merge 1 commit into nschloe:main from RemDelaporteMathurin:patch-1

RemDelaporteMathurin commented Sep 28, 2023

I recently tried to convert a mesh and ran into an assertion error in _float_to_nastran_string.

To reproduce the bug

from meshio.nastran._nastran import _float_to_nastran_string

_float_to_nastran_string(-1.5614246689128802e-18)

Traceback (most recent call last):
  File "/home/remidm/festim-review/comparison/achlys/convert_mesh.py", line 27, in <module>
    _float_to_nastran_string(-1.5614246689128802e-18)
  File "/home/remidm/miniconda3/envs/festim/lib/python3.11/site-packages/meshio/nastran/_nastran.py", line 416, in _float_to_nastran_string
    assert len(out) <= 16
           ^^^^^^^^^^^^^^
AssertionError

The value of out is -1.56142466891E-18 which is 18 characters long.

This PR reduces the precision argument in format_float_scientific to 9 and fixes the bug.

ping @gabriele-ferrero


          reduced precision to 9

8d8fef3

Author

RemDelaporteMathurin commented Sep 28, 2023

@nschloe I realise this may be a duplicate of #1350

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment