Profiler: use amrex logic for device synchronization instead of custom solution

[lucafedeli88]

This PR is a follow-up of the discussion with @AlexanderSinn in #4880 .

Currently we implement in ablastr some logic to ensure device synchronization for the TinyProfiler.
However, after @AlexanderSinn 's PR AMReX-Codes/amrex#3763 , we can rely on what is already implemented in AMReX in order to achieve that.

Therefore, this PR does the following:

• The AMReX logic is now used for device synchronization of the profiler. This is achieved by doing

 //https://github.com/AMReX-Codes/amrex/pull/3763
#ifdef AMREX_USE_GPU
        bool warpx_do_device_synchronize = true;
#else
        bool warpx_do_device_synchronize = false;
#endif
        pp_warpx.query("do_device_synchronize", warpx_do_device_synchronize);
        bool do_device_synchronize = warpx_do_device_synchronize;
        amrex::ParmParse pp_tiny_profiler("tiny_profiler");
        if (pp_tiny_profiler.queryAdd("device_synchronize_around_region", do_device_synchronize) )
        {
            WARPX_ALWAYS_ASSERT_WITH_MESSAGE(
                do_device_synchronize == warpx_do_device_synchronize,
                "tiny_profiler.device_synchronize_around_region overrides warpx.do_device_synchronize.");
        }

inside WarpXAMReXInit.cpp`.

• WarpXProfilerWrapper.H and ablastr/profiler/ProfilerWrapper.H are now ultra-thin wrappers around the macros provided by AMReX.

• WarpX::do_device_synchronize static variable is eliminated (with some minor changes to functions that required this variable as a parameter).

• All the other changes are inclusions of dependencies that were previously included transitively via #include "WarpX.H" inside WarpXProfilerWrapper.H.

[AlexanderSinn]

Looks Great!

[ax3l]

@lucafedeli88 thank you for this! Can you rebase pls? :)

[lucafedeli88]

@lucafedeli88 thank you for this! Can you rebase pls? :)

that's done, @ax3l !

[lucafedeli88]

ping, @AlexanderSinn and @ax3l ;-) !

[ax3l]

LGTM. This new default might hide a couple of race conditions that we introduce in the future 😅

@lucafedeli88 @AlexanderSinn , did you have a chance to double check performance stays the same?