Problems in setting nodal-type Multifab

[solarmaterial]

Hey AMReX team,
I am trying to transfer the data from the cell-centerd Multifab $\phi$ to the face type Multifab $\beta$, and I use MFIter to do so. Here is the code(phi_k is a cell-centerd Multifab). (1d calculation)

face_bcoef[0].FillBoundary(geom.periodicity());
Dim3 lox = lbound(geom.Domain()); 
Dim3 hix = ubound(geom.Domain());

for ( MFIter mfi(phi_k); mfi.isValid(); ++mfi )
    {
        const Box& xbx = mfi.nodaltilebox(0);
        auto const& face_bcoef_x = face_bcoef[0].array(mfi);  
        auto const& array_phi_k = phi_k.array(mfi);
     ParallelFor(xbx,
        [=] AMREX_GPU_DEVICE (int i, int j, int k)
        {
            if(i == hix.x)
            {face_bcoef_x(i,j,k) = (array_phi_k(i-2,j,k) + array_phi_k(i-1,j,k)) / 2;}
            else if(i == lox.x)
            {face_bcoef_x(i,j,k) = (array_phi_k(i,j,k) + array_phi_k(i+1,j,k)) / 2;}
            else
            {face_bcoef_x(i,j,k) = (array_phi_k(i-1,j,k) + array_phi_k(i,j,k)) / 2;}
        });
    }

Last time I set max_grid_size = n_cell =256 so that the face_bcoef[0] has no overlapping point, The code worked as I expected, and I also received some useful suggestions about the code. I really appreciate that!#3843 (comment) However, when I set something like max_grid_size =256, n_cell =128 ,the output of the face_bcoef[0] becomes:

#                       x                    comp0
                 0.015625                      2.5
                 0.046875                      2.5
                 0.078125                      3.5
                 0.109375                      4.5
......
                 0.453125                     15.5
                 0.484375                     16.5
                 0.515625                      nan
                 0.515625                      nan
                 0.546875                     18.5
                 0.578125                     19.5
......
                 0.953125                     31.5
                 0.984375                     31.5
                 1.015625                      nan

Actually, when max_grid_size = n_cell, there will be only one x = 0.515625 in the list, so I wonder how can I set the right number for the point of x = 0.515625, (the list have two points of x = 0.515625 is for parallel computing, I guess? so the values of comp0 at these two points should be the same as 17.5), and the value of comp0 at x = 0.984375 and x = 1.015625 should be 32.5. Still looking forward to your reply!

Yours sincerely,
solar

[ajnonaka]

How are you building face_bcoeff[0]? Does it have ghost cells? Is it defined nodally?
I am assuming your domain has physical size [0,1] - if that's the case you should get an x values at exactly 0, exactly 0.5, and exactly 1.0. Also your first value of x is greater than 0 and the last value you report is greater than 1, something seems off.
Also, hix might have a cell-centered index here, it really needs to be 256, not 255 (print it out) since you are trying to update a face on the upper-end of the domain.

[WeiqunZhang]

Besides what @ajnonaka has mentioned regarding what you are printing exactly, my best guess on how to fix it based on your previous questions is,

// assuming phi_k has at least one ghost cell                                                                                                                                
phi_k.FillBoundary(geom.periodicity());
Dim3 lox = lbound(amerx::convert(geom.Domain(),face_bcoef[0].ixType()));
Dim3 hix = ubound(amerx::convert(geom.Domain(),face_bcoef[0].ixType()));

for ( MFIter mfi(phi_k); mfi.isValid(); ++mfi )
{
    const Box& xbx = mfi.nodaltilebox(0);
    auto const& face_bcoef_x = face_bcoef[0].array(mfi);
    auto const& array_phi_k = phi_k.array(mfi);
    ParallelFor(xbx, [=] AMREX_GPU_DEVICE (int i, int j, int k)
    {
        if(i == hix.x) {
            face_bcoef_x(i,j,k) = array_phi_k(i-1,j,k);
        else if(i == lox.x) {
            face_bcoef_x(i,j,k) = array_phi_k(i,j,k);
        } else {
            face_bcoef_x(i,j,k) = (array_phi_k(i-1,j,k) + array_phi_k(i,j,k)) / 2;
        }
    });
}

[WeiqunZhang]

Also note that for the linear solver, the face based beta coefficient does not need any ghost cells.

[solarmaterial]

Thank you for your replies! @ajnonaka @WeiqunZhang I really appreciate that! Yes, the face_bcoeff[0] here is the face based beta coefficient of the ABecLaplacian linear solver, and it's nodal only in one dir, cell-centered in the other two dir( if 3D calculation), with no ghost cells. Meanwhile，the codes above do fix the issue!! It works! I guess I've got much to learn from you @WeiqunZhang, and thank you again for your kindly help!