error prepareTableListToUpload no such file or directory

[chummmaria]

Hi thanks for the lib!

I cannot upload the backup via sftp, when it has table with name like '%trouble%:

clickhouse-backup upload test
2024/01/16 20:30:00.551436 error b.prepareTableListToUpload return error: stat /clickhouse/backup/test/metadata/nko/%2Einner%2EPayments_trouble.json: no such file or directory

But the file exists:

ls -l /clickhouse/backup/test/metadata/nko/
total 232
-rw-r--r-- 1 clickhouse clickhouse 719 Jan 16 20:37 Payments_trouble.json

Backup is successfully unloaded if the table is added to skip_tables in the config.

Can I get around this error, but so that the table goes into the backup?

[Slach]

%2Einner%2EPayments_trouble.json means your payments_trouble table is MATERIALIZED VIEW? am i right?

looks like you face with
#765

try to insert data into Payments_troble

[Slach]

@olegtap
looks like your a_bds_line_item_native_view just doesn't contains data during backup
try to

clickhouse-backup restore --schema sp_pdata.a_bds_line_item_native_view backup_24-03-09

and wait when #765 resolve

[olegtap]

Actually, there are files in folder "%2Einner%2Ea_bds_line_item_native_view".
I downloaded and extracted from remote S3 backup manually.
Maybe I need to do something additionally for the tables like "%2Einner%2Ea_bds_line_item_native_view"?

[Slach]

@olegtap did you use standard upload or create_remote commands during backup creation and download command before restore?
or you use custom uploader, downloader
and use only create and restore command?

[Slach]

could you check

ls -la /var/lib/clickhouse/backup/backup_24-03-09/metadata/sp_pdata/

[olegtap]

The backup was done by create_remote to S3.
I manually downloaded it to the new server(my main goal to build a cluster)
and manually did "untar" to all the tar.gz files in /var/lib/clickhouse/backup/backup_2024-03-09/shadow/sp_data/.
Then I run clickhouse-backup restore --rm -t sp_data.%2Einner%2Ea_bds_line_item_native_view backup_2024-03-09
I did the same restore to the tables without ".inner." and it worked.

[root@CH-Cluster-01: /var/lib/clickhouse/backup/backup_2024-03-09/metadata/sp_data]# ls -la                                               
total 1772                                                                                                                                                                   
-rw-r--r--. 1 root root   2005 Mar 17 18:07 %2Einner%2Ea_bds_line_item_native_view.json                                                                          
-rw-r--r--. 1 root root   5760 Mar 17 18:07 %2Einner%2Ea_bds_line_item_view.json                                                                                             
-rw-r--r--. 1 root root  13209 Mar 17 18:07 %2Einner%2Ea_bidblock_buyer_view.json  

#######################################################################################3

[root@CH-Cluster-01: /var/lib/clickhouse/backup/backup_2024-03-09/metadata/sp_data]#  
                  cat %2Einner%2Ea_bds_line_item_native_view.json

{                                                                                                                                                                            
        "files": {        
                "default": [
                        "default_0_3_3_0.tar.gz",         
                        "default_16_6_6_0.tar.gz",
                        ...
                        "default_99_4_4_0.tar.gz"
                ]                                                                     
        },                
        "table": ".inner.a_bds_line_item_native_view",
        "database": "sp_data",                                                   
        "parts": {        
                "default": [
                        {                                                             
                                "name": "0_3_3_0"
                        },
                        {                                                             
                                "name": "16_6_6_0"
                        },
                        {
                                "name": "280_10166_10166_0"                                                                                                                                          
                        },                                                                                                                                     
     ...
      ]
        },
        "query": "CREATE TABLE sp_data.`.inner.a_bds_line_item_native_view` (`region_id` UInt8, `seller_account_id` Int64, `buyer_account_id` Nullable(Int64), `campaign_id` Nullable(Int64), `native_line_item_id` Int64, `impression` UInt64, `video_0` UInt64, `video_25` UInt64, `video_50` UInt64, `video_75` UInt64, `
video_100` UInt64, `decision_requests` Nullable(UInt64), `opportunity` Nullable(UInt64), `filled_adbreaks` Nullable(UInt64), `empty_adbreaks` Nullable(UInt64), `error` UInt6
4) ENGINE = MergeTree PARTITION BY seller_account_id ORDER BY seller_account_id SETTINGS index_granularity = 8192",
        "size": {
                "default": 47100749
        },
        "total_bytes": 47084129,
        "metadata_only": false
 }       

[Slach]

untar files default_XXX.tag.gz into /var/lib/clickhouse/backup/backup_2024-03-09/shadow/database/table/
or try
clickhouse-backup download backup_2024-03-09
instead of manually download + untar

[olegtap]

@Slach Thank you for your help.
Eventually, based on your advice in #613 (comment)
I decided to restore ".inner.table" in to the table with the real name and then recreate MaterializedView that will point to this new destination table.
I did the following in the backup folder:

1. In the main metadata.json I copied stanza with the ".inner.table" and replaced table name with the new name, like this

               {
                       "database": "sp_data",
                       "table": "dest_a_bds_line_item_native_view"
               },
               {
                       "database": "sp_data",
                       "table": ".inner.a_bds_line_item_native_view"
               },

2. In shadow folder I moved folder %2Einner%2Ea_bds_line_item_native_view to dest_a_bds_line_item_native_view.
3. In metadata folder I copied %2Einner%2Ea_bds_line_item_native_view.json to dest_a_bds_line_item_native_view.json.
4. In this file (dest_a_bds_line_item_native_view.json) I replaced .inner.a_bds_line_item_native_view with dest_a_bds_line_item_native_view.
5. I run restore with the new table name, and it worked successfully
   clickhouse-backup restore --rm -t sp_data.dest_a_bds_line_item_native_view backup_2024-03-09