service levels on raft: safe backup and restore

[piodul]

Service levels are going to be managed via raft in 6.0.

As we know from the case of schema-on-raft (#16349), restoring data via the usual restore procedure (link) to a raft-managed table is prone to issues if different sstables are uploaded to different nodes. It can happen for example if one backs up the same table on two nodes at a different point in time and the contents of the table changes. For raft tables, there is no manual or automatic (like in the case of schema) mechanism of repair, so they might be inconsistent until a manual intervention happens.

In case of schema, we decided to go with the approach of using a CQL command DESCRIBE SCHEMA WITH INTERNALS which returns a list of CQL statements that can be used to reconstruct the schema. We can consider doing the same for service levels: introduce a DESCRIBE SERVICE LEVELS command which generates a sequence of the right CREATE SERVICE_LEVEL and ATTACH SERVICE_LEVEL commands.

Considerations:

• Service levels are attached to roles, so they depend on auth. It should be possible to atomically dump service levels along with auth. Also see auth on raft: safe backup and restore #18750.