Support for OpenMC 0.13.0/HDF5 1.12.1

[yardasol]

Is your feature request related to a problem? Please describe.

The newest version of OpenMC, v0.13.0, was recently released. However, I have been unable to get PyNE to install alongside this version. Running conda install -c conda-forge openmc-env in an environment with PyNE v0.7.5 gives the following error:

Encountered problems while solving:
  - package pyne-0.7.5-moab_openmcpy39hcbc5909_0 requires hdf5 >=1.10.6,<1.10.7.0a0, but none of the providers can be installed

I have tried specifying the nomoab_noopenmc builds and run into the same problem.
I think this is happening due to all currently available builds of OpenMC v0.13.0 requiring HDF5 1.12.1 or higher. To my knowlege, all builds of PyNE 0.7.5 are pinned to HDF5 1.10.6.

Describe the solution or outcome you'd like.

Would it be possible to relax the HDF5 version requirement in PyNE? I'm not entirely sure what parts of HDF5 PyNE uses, so it's hard for me to say whether this is an easy thing to do in practice. I've looked around the PyNE repo for version files but haven't found any, so I wouldn't know where to start. I'd love to help though if this is a small change.

Describe alternatives you've considered.

The only alternative I can think of right now to get OpenMC v0.13.0 working with PyNE with is to build both OpenMC and PyNE from source on the same HDF5 version.

[welcome]

Hi, and welcome to PyNE! 👋 Thanks for opening your first issue. We recommend that you include information such as the version of PyNE you're working with (eg, develop branch or a specific version), the platform you are operating on, the expected behavior, and the actual behavior you are bringing our attention to. The more deatil you provide, the better others in this community will be able to help you.

[shimwell]

Just confirming this behavior

conda create --name openmc_with_pyne_test python=3.8
conda install -c conda-forge openmc=0.13.0 pyne

>>> UnsatisfiableError: The following specifications were found to be incompatible with each other:

Mamba install gives some hints regarding moab

Encountered problems while solving:
  - package pyne-0.7.1-moab_noopenmcpy38h033006a_0 requires libgfortran-ng >=7,<8.0a0, but none of the providers can be installed

The feedstock for pyne is here but there are not many version limits
https://github.com/conda-forge/pyne-feedstock/blob/master/recipe/meta.yaml

I can also confirm that even when installing without DAGMC in the OpenMC conda install there is the same conflict, this command also fails in the same way

mamba install -c conda-forge openmc=0.13.0=nodagmc* pyne

Lastly I can also confirm that the conda install of openmc v0.13.0 build 0 also doesn't install in a pyne environment. This is somewhat reassuring for me as it means this was a problem before I updated the openmc conda install a few days ago ☺️ this command also fails in the same way

mamba install -c conda-forge openmc=0.13.0=*81a01a0* pyne

[gonuke]

This appears to be an incompatibility at the GFortran layer. PyNE requires an older version of libgfortran and HDF 1.12 is built on conda-forge only with newer versions.

[yardasol]

I see. If HDF5 1.12 is built with a much newer version of gfortran I can see why that would cause incompatibility issues. Is the PyNE dependency on this older version of libgfortran only related to its HDF5 version, or are there other components of the code that rely on the older version of libgfortran?

[gonuke]

The PyNE dependency on FORTRAN is not related to HDF5. We might be able to build a conda package that is independent of FORTRAN.

[yardasol]

Oh is this related to ahot perhaps?

[yardasol]

The PyNE dependency on FORTRAN is not related to HDF5. We might be able to build a conda package that is independent of FORTRAN.

@gonuke any way I can help with this? I'd imagine this means modifying the feedstock in pyne-feedstock and adding some CMake options...

It looks like there may already be some machinery to do this in the setup.py file (assuming that the only Fortran dependencies are needed to build the spatialsolver module). On line 299:

if ns.spatial_solvers is not None:
    spatial_solvers = "ON" if ns.spatial_solvers else "OFF"
    ns.cmake_args.append("-DENABLE_SPATIAL_SOLVERS=" + spatial_solvers)

[gonuke]

Help is always welcome!!!

Turning off the spatial solvers is one part of this that already has a CMake option as you have found, but may not be the whole story.

There is also some FORTRAN in the ENSDF handlers and there is not yet an option to turn that off. Furthermore, I'm not entirely sure what the consequences of turning that off would be.

I'd be happy to have you try, however.

[yardasol]

I've been trying to build PyNE from source with the --no_spatial_solvers flag to test this out, but I'm running into the behavior described in #1416. I'm going to try to downgrade my version of GCC and G++ in the meantime so I can build from source

[yardasol]

Okay, I've gotten PyNE to build with OpenMC v0.13.0. Full gist of terminal output is here.
Versions:

• Python: 3.9
• HDF5: 1.12.1
• Using spatial solvers?: No

I'm running into an error when I'm running the test suite:

Testing test_enrichment.py:

....................malloc(): unaligned tcache chunk detected
travis-run-tests.sh: line 3: 25021 Aborted                 (core dumped) nosetests "$x"

I tried rebuilding using HDF5 1.10.6 and ran into the same error, so I don't think it's related to HDF5.

I've been able to track down the error to the creation of a Cascade object via enrichment.multicomponent, but that's as far as I've gotten so far.

Looks like to go farther into this I'll need to build python from source. There are two ways I'm thinking about doing this.

1. Build python with debugging symbols in the and then use the Cython debugger. It may be possible to do this with a pre-compiled version of python with debugging symbols.
2. Build python from source and then use tie python-gdb.py file (tutorial for this on Python Developer's Guide) with python-extended gdb.

Either way, I'll need to somehow get that custom build of python into the conda environment (example how to do this is here).

If there is an easier way around all this or someone knows the answer I'd love to hear it.

[yardasol]

I have installed Python from source sucessfully using the following steps:

1. Install the necessary dependencies (guide on Python Dev guide)
2. Download Python 3.9.10:

git clone git@github.com:python/CPython
cd cpython
git checkout tags/3.9.10 -b 3.9

4. Configure the build file and build CPython:

./configure --with-pydebug --with-cxx-main=gcc-9 CC=gcc-9 CXX=g++-9
make -s -j8

5. Test the build

./python -m test

All tests passed sucessfully. Now that I know this works, I'll need to get this into a Conda package.

Update: I've gotten the debug build of python into it's own package. I've added functionality to do this automatically in ubuntu-nuclear-software-installer.

Now I need to setup the Cython debugger. Will work on this tomorrow.

[yardasol]

Okay, I'm still going to try to set up the Cython debugger becase I think it could help demystify what's going on, but I was able to get a gbd backtrace for this malloc error:

$ gdb python test_enrichment.py
GNU gdb (Ubuntu 11.1-0ubuntu2) 11.1
Copyright (C) 2021 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from python...
"/home/ooblack/projects/pyne/tests/test_enrichment.py" is not a core dump: file format not recognized
(gdb) run test_enrichment.py
Starting program: /home/ooblack/miniconda3/envs/pyne-env/bin/python test_enrichment.py
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
/home/ooblack/miniconda3/envs/pyne-env/lib/python3.9/site-packages/nose/importer.py:12: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  from imp import find_module, load_module, acquire_lock, release_lock
[New Thread 0x7ffff40fe640 (LWP 15232)]
[New Thread 0x7ffff38fd640 (LWP 15233)]
[New Thread 0x7fffeb0fc640 (LWP 15234)]
[New Thread 0x7fffda8fb640 (LWP 15235)]
[New Thread 0x7fffd20fa640 (LWP 15236)]
[New Thread 0x7fffc98f9640 (LWP 15237)]
[New Thread 0x7fffc10f8640 (LWP 15238)]
[New Thread 0x7fffb88f7640 (LWP 15239)]
[New Thread 0x7fffb80f6640 (LWP 15240)]
[New Thread 0x7fffa78f5640 (LWP 15241)]
[New Thread 0x7fff9f0f4640 (LWP 15242)]
[New Thread 0x7fff90ea0640 (LWP 15243)]
[New Thread 0x7fff9069f640 (LWP 15244)]
[New Thread 0x7fff8fe9e640 (LWP 15245)]
[New Thread 0x7fff8f69d640 (LWP 15246)]
[New Thread 0x7fff8ee9c640 (LWP 15247)]
[New Thread 0x7fff8e69b640 (LWP 15248)]
[New Thread 0x7fff8de9a640 (LWP 15249)]
[New Thread 0x7fff8d699640 (LWP 15250)]
/home/ooblack/miniconda3/envs/pyne-env/lib/python3.9/site-packages/nose/suite.py:106: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working
  if isinstance(tests, collections.Callable) and not is_suite:
....................malloc(): unaligned tcache chunk detected

Thread 1 "python" received signal SIGABRT, Aborted.
__pthread_kill_implementation (no_tid=0, signo=6, threadid=140737350596416) at pthread_kill.c:44
44	pthread_kill.c: No such file or directory.
(gdb) bt
#0  __pthread_kill_implementation (no_tid=0, signo=6, threadid=140737350596416) at pthread_kill.c:44
#1  __pthread_kill_internal (signo=6, threadid=140737350596416) at pthread_kill.c:80
#2  __GI___pthread_kill (threadid=140737350596416, signo=signo@entry=6) at pthread_kill.c:91
#3  0x00007ffff7ce4476 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
#4  0x00007ffff7cca7b7 in __GI_abort () at abort.c:79
#5  0x00007ffff7d2b606 in __libc_message (action=action@entry=do_abort, fmt=fmt@entry=0x7ffff7e7d13d "%s\n") at ../sysdeps/posix/libc_fatal.c:155
#6  0x00007ffff7d42afc in malloc_printerr (str=str@entry=0x7ffff7e80330 "malloc(): unaligned tcache chunk detected") at malloc.c:5543
#7  0x00007ffff7d46cac in tcache_get (tc_idx=<optimized out>) at malloc.c:3082
#8  __GI___libc_malloc (bytes=48) at malloc.c:3200
#9  0x00007fff92385cbc in operator new(unsigned long) () from /lib/x86_64-linux-gnu/libstdc++.so.6
#10 0x00007fff9531540b in __gnu_cxx::new_allocator<std::_Rb_tree_node<std::pair<int const, double> > >::allocate(unsigned long, void const*) ()
    at /home/ooblack/projects/pyne/src/decay-linux-gnu.s:3569886
#11 0x00007fff953152c8 in std::_Rb_tree<int, std::pair<int const, double>, std::_Select1st<std::pair<int const, double> >, std::less<int>, std::allocator<std::pair<int const, double> > >::_M_get_node() () at /home/ooblack/projects/pyne/src/decay-linux-gnu.s:3569672
#12 0x00007fff953798f4 in std::_Rb_tree<int, std::pair<int const, double>, std::_Select1st<std::pair<int const, double> >, std::less<int>, std::allocator<std::pair<int const, double> > >::_M_create_node<std::pair<int const, double> const&> (this=0x7fffffff93f0) at /usr/include/c++/9/bits/stl_tree.h:630
#13 0x00007fff9537969c in std::_Rb_tree<int, std::pair<int const, double>, std::_Select1st<std::pair<int const, double> >, std::less<int>, std::allocator<std::pair<int const, double> > >::_Reuse_or_alloc_node::operator()<std::pair<int const, double> const&> (this=0x7fffffff90c0, __arg={...}) at /usr/include/c++/9/bits/stl_tree.h:491
#14 0x00007fff953794fc in std::_Rb_tree<int, std::pair<int const, double>, std::_Select1st<std::pair<int const, double> >, std::less<int>, std::allocator<std::pair<int const, double> > >::_M_clone_node<std::_Rb_tree<int, std::pair<int const, double>, std::_Select1st<std::pair<int const, double> >, std::less<int>, std::allocator<std::pair<int const, double> > >::_Reuse_or_alloc_node> (this=0x7fffffff93f0, __x=0x5555564fd270, __node_gen=...) at /usr/include/c++/9/bits/stl_tree.h:658
#15 0x00007fff953790d8 in std::_Rb_tree<int, std::pair<int const, double>, std::_Select1st<std::pair<int const, double> >, std::less<int>, std::allocator<std::pair<int const, double> > >::_M_copy<std::_Rb_tree<int, std::pair<int const, double>, std::_Select1st<std::pair<int const, double> >, std::less<int>, std::allocator<std::pair<int const, double> > >::_Reuse_or_alloc_node> (this=0x7fffffff93f0, __x=0x5555564fd270, __p=0x555556409e70, __node_gen=...) at /usr/include/c++/9/bits/stl_tree.h:1880
#16 0x00007fff95379118 in std::_Rb_tree<int, std::pair<int const, double>, std::_Select1st<std::pair<int const, double> >, std::less<int>, std::allocator<std::pair<int const, double> > >::_M_copy<std::_Rb_tree<int, std::pair<int const, double>, std::_Select1st<std::pair<int const, double> >, std::less<int>, std::allocator<std::pair<int const, double> > >::_Reuse_or_alloc_node> (this=0x7fffffff93f0, __x=0x5555564fd230, __p=0x7fffffff93f8, __node_gen=...) at /usr/include/c++/9/bits/stl_tree.h:1886
#17 0x00007fff95378cff in std::_Rb_tree<int, std::pair<int const, double>, std::_Select1st<std::pair<int const, double> >, std::less<int>, std::allocator<std::pair<int const, double> > >::_M_copy<std::_Rb_tree<int, std::pair<int const, double>, std::_Select1st<std::pair<int const, double> >, std::less<int>, std::allocator<std::pair<int const, double> > >::_Reuse_or_alloc_node> (this=0x7fffffff93f0, __x=..., __gen=...) at /usr/include/c++/9/bits/stl_tree.h:906
#18 0x00007fff953788a4 in std::_Rb_tree<int, std::pair<int const, double>, std::_Select1st<std::pair<int const, double> >, std::less<int>, std::allocator<std::pair<int const, double> > >::operator= (this=0x7fffffff93f0, __x=...) at /usr/include/c++/9/bits/stl_tree.h:1788
#19 0x00007fff953782ab in std::map<int, double, std::less<int>, std::allocator<std::pair<int const, double> > >::operator= (this=0x7fffffff93f0)
    at /usr/include/c++/9/bits/stl_map.h:100
#20 0x00007fff953782d9 in pyne::Material::operator= (this=0x7fffffff93f0) at /home/ooblack/projects/pyne/src/material.h:71
#21 0x00007fff9537864d in pyne::enrichment::Cascade::operator= (this=0x7fffffff9350) at /home/ooblack/projects/pyne/src/enrichment_cascade.h:21
#22 0x00007fff95376a1c in pyne::enrichment::multicomponent (orig_casc=..., solver="symbolic", tolerance=9.9999999999999994e-12, max_iter=100)
    at /home/ooblack/projects/pyne/src/enrichment.cpp:402
#23 0x00007fff8cadeb56 in __pyx_pf_4pyne_10enrichment_24multicomponent (__pyx_self=0x0, __pyx_v_orig_casc=0x7fff8c8c4a70, __pyx_v_solver=0x7ffff7444e80, 
    __pyx_v_tolerance=9.9999999999999994e-12, __pyx_v_max_iter=100) at /home/ooblack/projects/pyne/build/pyne/enrichment.cxx:8390
#24 0x00007fff8cade6bc in __pyx_pw_4pyne_10enrichment_25multicomponent (__pyx_self=0x0, __pyx_args=0x7fff8c8c7280, __pyx_kwds=0x7fff8c8c4770)
    at /home/ooblack/projects/pyne/build/pyne/enrichment.cxx:8288
#25 0x0000555555796bfc in cfunction_call (func=0x7fff8cafea10, args=<optimized out>, kwargs=<optimized out>) at Objects/methodobject.c:543
#26 0x00005555555c1dfe in _PyObject_MakeTpCall (tstate=tstate@entry=0x5555559b1510, callable=callable@entry=0x7fff8cafea10, args=args@entry=0x7fff8c8cd7b0, 
    nargs=<optimized out>, keywords=keywords@entry=0x7ffff7400950) at Objects/call.c:191
#27 0x0000555555686f5e in _PyObject_VectorcallTstate (kwnames=0x7ffff7400950, nargsf=9223372036854775809, args=0x7fff8c8cd7b0, callable=0x7fff8cafea10, tstate=0x5555559b1510)
    at ./Include/cpython/abstract.h:116
#28 PyObject_Vectorcall (kwnames=0x7ffff7400950, narg

@gonuke See anything of note?

[yardasol]

@gonuke any ideas?

[gonuke]

We now have a successful build of pyne with newer OpenMC/HDF5 (#1470), so we should consider a release and update of our conda-forge image.