cgroups memory limit does not cover memory allocated by ROCm hipMalloc()

[garlick]

Problem: Memory allocated by ROCm hipMalloc() is not subject to memory limits.

When flux runs with

[systemd]
enable = true
[exec]
service = "sdexec"
[exec.sdexec-properties]
MemoryMax = "95%"

We observed that memory allocated by hipMalloc() is not accounted for in the job's cgroup, and the user is able to exceed MemoryMax and run the kernel out of memory.

An issue was opened with AMD (though not public at this time).