Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

restore command for ReplicatedMergeTree with --restore-database-mapping, should check replication path is exists in ZK and generate new path with predefined format, i.e. /clickhouse/tables/{cluster}/{shard}/{database}/{table} #849

Open
Slach opened this issue Feb 22, 2024 · 6 comments
Assignees
Milestone

Comments

@Slach
Copy link
Collaborator

Slach commented Feb 22, 2024

No description provided.

@Slach Slach added this to the 2.5.0 milestone Feb 22, 2024
@Slach Slach modified the milestones: 2.5.0, 2.6.0 Apr 8, 2024
@supnobita
Copy link

supnobita commented Apr 19, 2024

Hi @Slach,
When i restored a table on a clickhouse cluster, I used --restore-database-mapping after that I saw clickhouse-backup create a different uuid for the same table on ZK, so that the table could not sync. Did it relate to this issue?

@Slach
Copy link
Collaborator Author

Slach commented Apr 19, 2024

no this is a different issue, but could be related

usign {uuid} in replication path always bad idea from my perspective

do you mean restoring ReplicatedMergeTree on the different replicas with --restore-database-mapping replace {uuid} with different values on different replicas?

as a workaround you could change manually /var/lib/clickhouse/backup/backup_name/metadata/db/table.json
and replace ReplicatedMergeTree('/path/{uuid}','{replica}') to ReplicatedMergeTree('/path/{shard}/{database}/{table}','{replica}')

@supnobita
Copy link

do you mean restoring ReplicatedMergeTree on the different replicas with --restore-database-mapping replace {uuid} with different values on different replicas? Yes ! And the new uuid(s) are different with the uuid in table.json in the backup metadata. That mean Clickhouse-backup create a new uuid, it didn't use the uuid of the backup.

I will try the workaround tmr. Thanks

@Slach
Copy link
Collaborator Author

Slach commented Apr 19, 2024

you also could try to use

general:
 restore_metadata_on_cluster: "{cluster}"

after that ReplicatedMergeTree will create on cluster with the same UUID

it didn't use the uuid of the backup.

uuid is not related to backup
this is clickhouse-server behavior when you created engine=ReplicatedMergeTree() without parameter the table will create with default_replica_path (which contains {uuid} and default_replica_path values

@Slach
Copy link
Collaborator Author

Slach commented Apr 19, 2024

if you will use restore_metadata_on_cluster
in this case you just need execute restore --schema only in one replica in whole cluster

and use restore --data only in first replica in each shard

@Slach Slach self-assigned this Apr 21, 2024
@supnobita
Copy link

It worked !

Thank you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants