Decentralized ML framework

[amirjaber]

Is your feature request related to a problem? Please describe.
We are a team of developers building a decentralized blockchain based ML framework. In this work, multiple machines accross internet should connect and send files to each other in each round of training. The head node responsible for aggregating the files that workers has passed to it. Head node should be switched every iteration to make it decentralized.

Describe the solution you'd like
We need a message passing mechanism that provide the IP and status of each node in the network at each iteration. Depending on those information the computers decide which computer to communicate with and send their training files to.

Describe alternatives you've considered
We have used http protocol and json files successfully for this communication.

Additional context
We are looking into any message passing library that can work with many computers and has the the feature to update the head node and workers IP to make it decentralized.