AMD EPYC 7K62 NCCL-test 4090 bandwidth too

[ghoul02015]

CPU：AMD EPYC 7K62
mem：32G *16
AMD EPYC Why is p2p performance so slow? Is there anything to set

[sjeaugey]

You may try to set NCCL_P2P_LEVEL=PXB and see whether it works better.

Note that p2pBandwidth only creates traffic between 2 GPUs at once. Performance tends to degrade significantly when we have traffic going from/to all GPUs and interleaving in the CPU. It would also be interesting to see what performance you get with 2 GPUs only (which would be kind of equivalent to the p2pBandwidth test in bidirectional mode and --sm_copy mode).