Docs/Compositing: composite cluster usecases. #278
Conversation
ghost
added
the
There was a problem hiding this comment.
@hsanjuan Overall this is great. I see the value of composite clusters more clearly already. I don't have any answers right now regarding the many open questions you've posed but they are now on my mind. Not sure if it belongs in this document but an important next step is figuring out exactly what to do next with these ideas. I am thinking we will need to nail down enough questions in "The IPFS-like cluster API" to know what we are are building and we will need to get a sense of the answers to your useability questions under "Administrative overhead" in order to select a use case that composite clusters can significantly impact so that we can build towards something concrete. Of course this is just my first take away.
docs/composite-clusters.md
Outdated
|
|
||
| ## Introduction | ||
|
|
||
| The idea of composite ipfs-clusters is based on the capacity of a cluster node to provide an ipfs-proxy endpoint in which HTTP requests are partially hijacked and used to trigger cluster operations or retrieve cluster information. This means that ipfs-clusters offers a partial ipfs-compatible API. |
There was a problem hiding this comment.
I would consider saying "partial IPFS API", it's not about cluster implementing an api compatible with ipfs use, it's about cluster implementing the same api as ipfs.
|
|
||
| #### Uses | ||
|
|
||
| ##### Destroy the world |
There was a problem hiding this comment.
Maybe consider adding a figure here as well:
For example
docs/composite-clusters.md
Outdated
|
|
||
| * Overcluster: the cluster at the top of a cluster composition topology | ||
| * Undercluster(s): the clusters at the bottom of a cluster composition topology | ||
| * "Peer attached to a peer": A cluster peer that uses the ipfs-proxy-endpoint provided by another peer as its IPFS daemon address. |
There was a problem hiding this comment.
I like the over and under terminology. It feels a bit off to call the nodes in the undercluster the "peers" of the nodes in the overcluster so I would go with something like "provided by another cluster daemon" and "peer attached to a cluster". Though perhaps I have the wrong idea about the relationship of cluster nodes in over and underclusters.
| * Undercluster(s): the clusters at the bottom of a cluster composition topology | ||
| * "Peer attached to a peer": A cluster peer that uses the ipfs-proxy-endpoint provided by another peer as its IPFS daemon address. | ||
|
|
||
| ## Topologies |
There was a problem hiding this comment.
The document is well organized for understanding.
docs/composite-clusters.md
Outdated
|
|
||
| The most straightforward cluster composition topology consists of a regular ipfs-cluster where the IPFS-endpoints for each cluster peer point to underlying, separate ipfs-clusters. | ||
|
|
||
| Todo: add drawing |
There was a problem hiding this comment.
Not super pretty but here's a sketch you could use that gets the point across
There was a problem hiding this comment.
I'm thinking if the ipfs daemons should be represented too?
docs/composite-clusters.md
Outdated
|
|
||
| In this topology, each cluster peers uses as ipfs daemon another cluster peer from a different cluster, but all underlying peers belong to the same cluster. | ||
|
|
||
| Todo: drawing |
docs/composite-clusters.md
Outdated
|
|
||
| ##### Data partitioning | ||
|
|
||
| A group of parties may which to share a common cluster. With an inverse multi-ring setup, each party will mantain their own pinset in their own cluster, but it will use a common infrastructure (the undercluster), which is shared with others. Each overcluster may administer how cluster APIs are exposed and what authentication to use. |
There was a problem hiding this comment.
I am having a hard time understanding why this is something people would want to do. In what situation would this be desirable over everyone banding together into a single cluster?
There was a problem hiding this comment.
Basically, maintaining a separate pinset, not worrying about ipfs setup, and being able to take down a cluster without affecting other people/clusters. Of course, most of it is just delegating problems to the person maintaining the under cluster.
|
|
||
| ### Administration overhead | ||
|
|
||
| * How hard is it to manage a composite cluster vs. managing a single one? Is it worth it to the use-cases? |
There was a problem hiding this comment.
👍 This is going to be key to figure out when planning which features to tackle with composite approaches.
|
|
||
| * How hard is it to manage a composite cluster vs. managing a single one? Is it worth it to the use-cases? | ||
| * What happens when there are errors, how are they handled in the different levels? Is the handling appropiate? | ||
| * Are there alternative ways of handling usecases, like pubsub? Are they better? |
There was a problem hiding this comment.
A lot of these usecases could potentially be addressed by building features into simple clusters, of course doing so might be a lot harder.
| This is a specific case of **inverse multi-ring topology** described below, with a single overcluster. | ||
|
|
||
| ### Multiple clusters have their cluster peers attached to peers from a single cluster (inverse multi-ring) | ||
|
|
Adding some thoughts about how composite clusters can be organized and what to do with that. License: MIT Signed-off-by: Hector Sanjuan <code@hector.link>
License: MIT Signed-off-by: Hector Sanjuan <code@hector.link>
hsanjuan
force-pushed
the
doc/composite-cluster-usecases
branch
from
40f515c
to
fca83cc
Compare
|
There were the following issues with your Pull Request
Code contribution guidelines for IPFS Cluster are available at https://cluster.ipfs.io/developer/contribute/#code-contribution-guidelines This message was auto-generated by https://gitcop.com |
|
We need to move this to the website docs |
Adding some thoughts about how composite clusters
can be organized and what to do with that.
License: MIT
Signed-off-by: Hector Sanjuan code@hector.link