Preferred Umpire version?

[pmaginot-lanl]

Does hypre have a preferred version of Umpire to use? Is Umpire v2023.06.0 supported and tested for integration into hypre, or via inference from the most recent related hypre merge looking like umpire2022 , should I target Umpire v.2022.03.1 instead?

[victorapm]

@pmaginot-lanl Sorry for the delay!

Does hypre have a preferred version of Umpire to use?

We don't have a preferred version AFAIK, but anything newer than v5.0.1 should work.

Is Umpire v2023.06.0 supported and tested for integration into hypre,

It is supported (I verified), although we are only testing our integration with v5.0.1 for now.

should I target Umpire v.2022.03.1 instead?

Umpire v2022.03.1 works, but as said above, It is also fine to use their newest build (v2023.06.0).

Please, let me know if you encounter any issues.

[pmaginot-lanl]

No issues, but alas not the panacea to issues we are encountering when using the IJ interface + GPU + Boomer.

Building hypre with GPU support and calling HYPRE_SetMemoryLocation(HYPRE_MEMORY_DEVICE); of hypre 2.29 results in an out of memory like error. With a debug build, we hit this assertion on the first try we attempt to solve a system of equations:

[par_csr_matvec_device.c, 136] hypre_assert failed: num_cols_offd * num_vectors == hypre_ParCSRCommPkgRecvVecStart(comm_pkg, num_recvs)
xrage.x: par_csr_matvec_device.c:136: HYPRE_Int hypre_ParCSRMatrixMatvecOutOfPlaceDevice(HYPRE_Complex, hypre_ParCSRMatrix*, hypre_ParVector*, HYPRE_Complex, hypre_ParVector*, hypre_ParVector*): Assertion `0' failed.

If we set HYPRE_SetMemoryLocation(HYPRE_MEMORY_HOST); instead, our use of hypre proceeds as desired. I believe Bobby Philip is working with Ulrike to diagnose and fix, but if I can be of any assistance please let me know.

The documentation appears to suggest that only the sstruct paths are RAJA-fied, but if building hypre with RAJA support gets us any closer to the code path that Ares uses, xRAGE would be happy to try it.

[victorapm]

Sounds good! Do you also have HYPRE_SetExecutionPolicy(HYPRE_EXEC_DEVICE); in your code?

[pmaginot-lanl]

Yep, on the next line

[victorapm]

If you have a chance to debug hypre, I suspect that assertion is failing because num_vectors != 1. If that's the case, one idea is to make sure the right/left-hand-side vectors are being created correctly. Hope this helps!