[META] Cluster State Publication using Remote State

[soosinha]

Please describe the end goal of this project

For remote store enabled clusters, the cluster state is being published to a remote repository. This can be used to publish the cluster state to all nodes in the cluster. Instead of sending the cluster state object over the transport layer, the cluster manager node can just send the term and version of the cluster state over the transport layer which can then be used to download the corresponding cluster state by all the nodes directly from the remote store.

Supporting References

RFC: #13257

Issues

• [Remote Cluster State] Add toXContent and fromXContent for ClusterBlocks #13692
• [Remote Cluster State] Add toXContent and fromXContent for DiscoveryNodes #13691
• [Remote Cluster State] Add XContent serde for ClusterState.Custom #13727
• [Remote Cluster State] Download cluster state on follower nodes #13682
• [Remote Cluster State] Upload ephemeral cluster state objects(cluster blocks, nodes, customs) to remote #13728
• [Remote Cluster State] Download the cluster state objects asynchronously and in parallel #13729
• [Remote Cluster State] Add new transport call to send cluster state term and version #13730
• [Remote Cluster State] Create interfaces for remote cluster state #13791
• [Remote Cluster State] Refactor RemoteClusterStateService to small decomposable units for each file write #12881

Related component

Cluster Manager

[peternied]

[Triage - attendees 1 2 3 4 5 6 7 8]
@soosinha Thanks for creating this issue

[lukas-vlcek]

Is #13576 already part of this effort? I just want to make sure it is linked if it is related (and possible listed in the Issues list).

[shiv0408]

@lukas-vlcek Yes, that PR is required for this. Created a new issue #13691 as a task for this.
@soosinha can you please help in adding this issue to the task list of this Meta issue.