[BUG]: thrust::count_if and copy_if performance on Grace and x86 10x+ / 20x+ slower than libstdc++

[gonzalobg]

Is this a duplicate?

• I confirmed there appear to be no duplicate issues for this bug and that I agree to the Code of Conduct

Type of Bug

Performance

Component

Thrust

Describe the bug

See title, the performance of count_if and copy_if are very low on NVIDIA Grace CPU.

How to Reproduce

This self-contained file reproduces: https://github.com/gonzalobg/cpp_hpc_tutorial/blob/master/labs/cpp/lab1_select/solutions/exercise1.cpp

This one isolates copy_if: https://github.com/gonzalobg/cpp_hpc_tutorial/blob/master/labs/cpp/lab1_select/solutions/copy_if.cpp

The performance of transform_inclusive_scan is also quite low (~3x lower) than libstdc++, see
https://github.com/gonzalobg/cpp_hpc_tutorial/blob/master/labs/cpp/lab1_select/solutions/exercise2.cpp

Expected behavior

Close to peak CPU bandwidth.

Reproduction link

No response

Operating System

No response

nvidia-smi output

No response

NVCC version

No response