State redistribution - areafrac out of bounds memory access

[andreasrauch]

Hello,

I am implementing state redistribution for a cut-cell EB method into a compressible flow solver that utilizes AMReX. I'm using the AMReX-Hydro Redistribution tools for this, but am encountering a SegFault due to out of bounds memory access in the function Redistribution::MakeITracker when calculating the normals from apx and apy (this is a 2D test case). The error output is below.

Apparently the bounding box of apx and apy is not the same as flag_arr (the EBCellFlags) which then leads to the segfault. Am I assigning apx and apy incorrectly? Attached the Backtrace files, the code output and a minimal version of the relevant source code where I call Redistribution::Apply and assign EB data structures.

`convSystem: just before Redistribution

flag_arr x bounds: -5,100
flag_arr y bounds: 59,100
apx x bounds: -2,98
apx y bounds: 62,97
IN MAKE_ITRACKER DOING BOX ((0,64) (95,95) (0,0))
MakeITracker: bxg4 = ((-4,60) (99,99) (0,0))
MakeITracker: bx_per_g4 = ((0,60) (95,95) (0,0))
MakeITracker: just before bx_per_g4 loop
MakeITracker: bx_per_g4_loop: i,j,k = 14,60,0
MakeITracker: apnorm, apnorminv declared
amrex::Abort::0:: (15,60,0,0) is out of bound (-2:98,62:97,0:0,0:0) !!!
SIGABRT
amrex::Abort::1:: (79,67,0,0) is out of bound (-2:98,-2:65,0:0,0:0) !!!
SIGABRT
See Backtrace.0 file for details
See Backtrace.1 file for details`

convSystem_minimal.txt
test_stateredist.txt
Backtrace.1.txt
Backtrace.0.txt

[WeiqunZhang]

The issue is probably how EBFArrayBoxFactory is built. You can tell it to have more ghost cells in area fractions. See https://amrex-codes.github.io/amrex/docs_html/EB.html?highlight=makeebfabfactory#ebfarrayboxfactory. The incflo codes uess {4,5,4} ghost cells. Maybe you can try that. The AMReX-Hydro code probably should also add an assertion with an error message.

[andreasrauch]

In the code I don't call makeEBFabFactory but define a MultiFab as:

Sborder.define(grids, dmap, NUM_STATE, NUM_GROW, MFInfo(), Factory());

Is this a correct way to build the EBFArrayBoxFactory?

Then I get the factory and area fractions as:

auto const& factory = dynamic_cast<EBFArrayBoxFactory const&>(S_border.Factory());
const auto&               areafrac    = factory.getAreaFrac(); // cut face area fraction

And finally apx, apy inside an MFIter loop as:

            AMREX_D_TERM(Array4<Real const> const& apx = areafrac[0]->const_array(mfi);,
                         Array4<Real const> const& apy = areafrac[1]->const_array(mfi);,
                         Array4<Real const> const& apz = areafrac[2]->const_array(mfi));

I tried the following, but it gave a compiler error:

Sborder.define(grids, dmap, NUM_STATE, {NUM_GROW, NUM_GROW, NUM_GROW}, MFInfo(), Factory());

[WeiqunZhang]

Okay, so you are using AmrLevel. In that case, somewhere in the initialization of the code, you can call the following.

AmrLevel::SetEBMaxGrowCells(4,5,4);

You can `git grep SetEBMax" in amrex/ to find some examples.

[andreasrauch]

That fixed it, thank you very much!