mca/coll: Add any radix k for alltoall bruck algorithm #12453
Conversation
| if (err != MPI_SUCCESS) { line = __LINE__; goto err_hndl; } | ||
|
|
||
| /* Sendreceive */ | ||
| err = ompi_coll_base_sendrecv ( tmpbuf, 1, new_ddt, sendto, |
There was a problem hiding this comment.
these can be replaced with non-blocking send/recv for better performance
There was a problem hiding this comment.
I tried replacing with non-blocking send/recv but it segfaults in osu_alltoall
There was a problem hiding this comment.
You won't get any performance buff with large fanout if you are using blocking send/receives. We can setup a meeting next week to see why it crashes.
There was a problem hiding this comment.
You do get some improvement because you decrease the depth of the tree, but you don't get the multi-rail benefit.
Talking about multi-rail, I would expect the p2p communication layer to take advantage of the multi-rail, at least after a certain message size. If that's the case when the upper and lower level of the communication stack both try to generate multi-rail traffic things cant go well. How do we protect against that ? Is the plan to only enable collectives multi-rail only for some ranges of messages ?
There was a problem hiding this comment.
Jiaxiyan is on an international trip. I will follow up with her to resolve the crash she mentioned.
|
I was planning to review it after they figure out the issue with replacing the blocking ompi_coll_base_sendrecv with non-blocking communications. |
This method extends ompi_coll_base_alltoall_intra_bruck to handle any radix k. Signed-off-by: Jessie Yang <jiaxiyan@amazon.com>
|
@bosilca I don't follow this comment:
Are you suggesting that collectives shouldn't be making PML isend/irecv directly? |
|
No, I was wondering how would a multi-rail enabled collective component (which will indeed post multiple isend requests) interacts with a multi-rail enabled low-level communication library (which will split all large messages across multiple lanes) ? |
This method extends ompi_coll_base_alltoall_intra_bruck to handle any radix k.