Ascent: SoA Particle Support #3350

ax3l · 2023-06-06T06:25:06Z

Summary

Add support for pure SoA layouted particle containers for Ascent.

Additional background

Follow-up to #2878.

Checklist

The proposed changes:

fix a bug or incorrect behavior in AMReX
add new capabilities to AMReX
changes answers in the test suite to more than roundoff level
are likely to significantly affect the results of downstream AMReX users
include documentation in the code and/or rst files, if appropriate

ax3l · 2023-08-10T22:01:11Z

@atmyers @c-wetterer-nelson @cyrush @andrewcombs let's coordinate how we finalize this - this is the last element blocking the AMReX SoA transition for WarpX on our agenda 🥁

c-wetterer-nelson · 2023-08-11T16:49:15Z

@atmyers @c-wetterer-nelson @cyrush let's coordinate how we finalize this - this is the last element blocking the AMReX SoA transition for WarpX on our agenda 🥁

RE: the catalyst 2 support PR. Yesterday was Andrew's last day so I will take over. The last piece there is a GitHub action script to add catalyst build testing to the CI. We could do that in a separate PR if you're comfortable with that. I am hoping to find time next week to make that happen. Otherwise, we are ready to review that PR.

cyrush · 2023-08-11T17:21:02Z

Src/Extern/Conduit/AMReX_Conduit_Blueprint_ParticlesI.H

@@ -32,7 +31,7 @@ ParticleTileToBlueprint(const ParticleTile<amrex::Particle<NStructReal,
                        const std::string &topology_name)
 {
    int num_particles = ptile.GetArrayOfStructs().size();
-    int struct_size   = sizeof(Particle<NStructReal, NStructInt>);
+    int struct_size   = ParticleType::is_soa_particle ? 0 : sizeof(ParticleType);


struct_size is used to represent element striding in the Conduit tree.

For the SOA case - I expect we want native striding (for example, 8 bytes for float64), so I think we need to update the Particle Container to Conduit wrappers to handle these cases.

Thank you Cyrus - where is that wrapper located? I thought it was this file...

cyrush · 2023-08-12T17:46:58Z

Src/Extern/Conduit/AMReX_Conduit_Blueprint_ParticlesI.H

 #endif
+    }


Yes, this is the style of logic we need.

If AOS, we stride by the struct size. If SOA, we stride by the native stride (set_external w/o the stride argument accomplishes this)

cyrush · 2023-08-12T17:48:15Z

Src/Extern/Conduit/AMReX_Conduit_Blueprint_ParticlesI.H

-    // struct real fields, the first set are always the particle positions
-    // which we wrap above
-    for (int i = 0; i < NStructReal; i++)
+    if constexpr(!ParticleType::is_soa_particle)


we need to build the fields for the SOA case, not sure how to access from AMReX but it should be a an else case.

ax3l · 2023-11-20T19:13:25Z

Restarting CI.

From local tests, I think CI does not catch that his PR still has a compile error: #3639

ax3l · 2023-11-20T20:03:50Z

Based on .github/workflows/ascent.yml and #3639, testing locally with:

docker run -v $PWD:/opt/amrex -it alpinedav/ascent:0.9.2

export CXX=g++
export CC=gcc
export CMAKE_PREFIX_PATH=/ascent/install/lib/cmake/

. /ascent_docker_setup_env.sh

cd /opt/amrex/
cmake -S . -B build          \
           -DCMAKE_BUILD_TYPE=RelWithDebInfo \
           -DAMReX_ENABLE_TESTS=ON  \
           -DAMReX_FORTRAN=OFF      \
           -DAMReX_ASCENT=ON   \
           -DAMReX_CONDUIT=ON  \
           -DAMReX_PARTICLES=ON
cmake --build build -j 8

@cyrush

## Summary For some reason, `AMReX_ASCENT` does not trigger `AMReX_CONDUIT` to be set, too. cc @cyrush ## Additional background X-ref: #3350

Add support for pure SoA layouted particle containers for Ascent.

Co-authored-by: Andrew Myers <atmyers2@gmail.com>

ax3l · 2023-11-21T00:02:45Z

I think our main.cpp is not yet complete :)

@cyrush

## Summary For some reason, `AMReX_ASCENT` does not trigger `AMReX_CONDUIT` to be set, too. cc @cyrush ## Additional background X-ref: AMReX-Codes#3350

## Summary Add support for pure SoA layouted particle containers for Ascent. ## Additional background Follow-up to AMReX-Codes#2878. ## Checklist The proposed changes: - [ ] fix a bug or incorrect behavior in AMReX - [x] add new capabilities to AMReX - [ ] changes answers in the test suite to more than roundoff level - [ ] are likely to significantly affect the results of downstream AMReX users - [ ] include documentation in the code and/or rst files, if appropriate --------- Co-authored-by: Andrew Myers <atmyers2@gmail.com>

ax3l added the help wanted label Jun 6, 2023

ax3l mentioned this pull request Jun 6, 2023

Update Particle Container to Pure SoA ECP-WarpX/WarpX#3850

Merged

12 tasks

This was referenced Jul 24, 2023

SENSEI: SoA Particle Support #3349

Merged

Added Catalyst2 Build Support #3444

Open

ax3l force-pushed the topic-soa-ascent branch from ebf8b95 to fc7f679 Compare July 25, 2023 01:57

ax3l requested a review from atmyers July 25, 2023 01:57

ax3l force-pushed the topic-soa-ascent branch from fc7f679 to a82c673 Compare July 27, 2023 20:20

cyrush reviewed Aug 11, 2023

View reviewed changes

ax3l force-pushed the topic-soa-ascent branch from a82c673 to ff66bed Compare August 12, 2023 01:45

cyrush reviewed Aug 12, 2023

View reviewed changes

ax3l assigned atmyers Oct 4, 2023

ax3l changed the title ~~[WIP] Ascent: SoA Particle Support~~ Ascent: SoA Particle Support Oct 4, 2023

cyrush approved these changes Oct 18, 2023

View reviewed changes

ax3l closed this Nov 20, 2023

ax3l reopened this Nov 20, 2023

ax3l mentioned this pull request Nov 20, 2023

CI: Ascent + Conduit #3639

Merged

5 tasks

WeiqunZhang pushed a commit that referenced this pull request Nov 20, 2023

CI: Ascent + Conduit (#3639)

c51e0a3

## Summary For some reason, `AMReX_ASCENT` does not trigger `AMReX_CONDUIT` to be set, too. cc @cyrush ## Additional background X-ref: #3350

ax3l and others added 5 commits November 20, 2023 15:23

Ascent: SoA Particle Support

07db32b

Add support for pure SoA layouted particle containers for Ascent.

Draft: Shamelessly Copy A Test

1af5b0a

Co-authored-by: Andrew Myers <atmyers2@gmail.com>

More Hackery

62ea9bc

some fixes for pure SoA with Ascent

9ac1bcc

SoA: Fix a Few Compile Errors

b178ad9

ax3l force-pushed the topic-soa-ascent branch from 4992cfe to b178ad9 Compare November 20, 2023 23:23

Test Ascent_Insitu_SOA Compiles

433db1e

Compile Fix: Legacy AoS

22b4141

atmyers approved these changes Nov 21, 2023

View reviewed changes

atmyers merged commit 9e35dc1 into AMReX-Codes:development Nov 21, 2023
69 checks passed

ax3l deleted the topic-soa-ascent branch November 27, 2023 18:00

guj pushed a commit to guj/amrex that referenced this pull request Dec 13, 2023

CI: Ascent + Conduit (AMReX-Codes#3639)

839836f

## Summary For some reason, `AMReX_ASCENT` does not trigger `AMReX_CONDUIT` to be set, too. cc @cyrush ## Additional background X-ref: AMReX-Codes#3350

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Ascent: SoA Particle Support #3350

Ascent: SoA Particle Support #3350

ax3l commented Jun 6, 2023

ax3l commented Aug 10, 2023 •

edited

c-wetterer-nelson commented Aug 11, 2023

cyrush Aug 11, 2023

ax3l Aug 12, 2023

cyrush Aug 12, 2023

cyrush Aug 12, 2023

ax3l commented Nov 20, 2023 •

edited

ax3l commented Nov 20, 2023

ax3l commented Nov 21, 2023

Ascent: SoA Particle Support #3350

Ascent: SoA Particle Support #3350

Conversation

ax3l commented Jun 6, 2023

Summary

Additional background

Checklist

ax3l commented Aug 10, 2023 • edited

c-wetterer-nelson commented Aug 11, 2023

cyrush Aug 11, 2023

Choose a reason for hiding this comment

ax3l Aug 12, 2023

Choose a reason for hiding this comment

cyrush Aug 12, 2023

Choose a reason for hiding this comment

cyrush Aug 12, 2023

Choose a reason for hiding this comment

ax3l commented Nov 20, 2023 • edited

ax3l commented Nov 20, 2023

ax3l commented Nov 21, 2023

ax3l commented Aug 10, 2023 •

edited

ax3l commented Nov 20, 2023 •

edited