You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
A thought, and part of a longer-term goal: can we implement an inter-node communication block? That is, could we implement an ethernet-based "virtual ring" that packetizes data up and sends it over from one computer to another? Or more explicitly have a "to another node" packetizer block, and a matching "from another node" depacketizer block?
Relatedly, is there an elegant way to launch and monitor pipelines on multiple systems? For example, when we run the LEDA pipeline we spin up 22 pipelines across 11 computers; if bifrost could facilitate pipeline bringup across nodes it would be great.
While these functions would be pretty darn neat, it would also be a massive time sink. But some thought now might make it easier in the future?
The text was updated successfully, but these errors were encountered:
I've been thinking about the inter-node communication aspect with the Verbs/RDMA work. I've started on a rdma.py module that should enable sending the contents of a ring, including when a sequence changes, in an "easy" way. The implementation leaves all of the packetization/depacketization details up to RDMA.
A thought, and part of a longer-term goal: can we implement an inter-node communication block? That is, could we implement an ethernet-based "virtual ring" that packetizes data up and sends it over from one computer to another? Or more explicitly have a "to another node" packetizer block, and a matching "from another node" depacketizer block?
Relatedly, is there an elegant way to launch and monitor pipelines on multiple systems? For example, when we run the LEDA pipeline we spin up 22 pipelines across 11 computers; if bifrost could facilitate pipeline bringup across nodes it would be great.
While these functions would be pretty darn neat, it would also be a massive time sink. But some thought now might make it easier in the future?
The text was updated successfully, but these errors were encountered: