dump requires absolute path from within the repo #4443

marbx · 2023-08-09T09:10:07Z

What does this PR change? What problem does it solve?

Dumping a file following the present documentation resulted in the error message given below.

Was the change previously discussed in an issue or on the forum?

I found other discussions where dumping failed, but not sure if they are related.

To be discussed:

Under which conditions the "Directory" (attached) path and the "repository" path differ?
Under which conditions dump requires the "Directory" path?
How to refer to chapters from the documentation? (currently plain hyperlinks)

MichaelEischer · 2023-08-13T17:36:05Z

Under which conditions the "Directory" path and the "repository" path differ?

The paths within a snapshot may differ from the attached path list if relative paths were used for the backup and to a certain extent for Windows paths.

For example restic backup work run from the folder /home/user would result in a snapshot with a path list of /home/user/work that will only contain /work. This path-related discrepancy applies to each command that tries to access data within a snapshot.

I wonder whether it would make sense to improve the description around how backup handles relative paths, instead of describing workarounds in some command section.

marbx · 2023-08-16T10:51:46Z

You clarified the discrepancy between the two paths very well and I used it to improve the backup command.

I recommend to improve not only the backup command, but dump and listing (restic snapshots) too:

Backup, dump and listing are separate use cases, they can be performed at different times or by different end users. Without a reference to the backup commands, an end user failing to use dump and unaware of the discrepancy, would need to read the whole documentation.
Including error messages in user documentation allows for searching and googling by error message.

I am not sure what you mean with "workaround". If you refer to the ls latest / command, I suggest to document how to look up the path within the repository: the path is necessary information, making the lookup a necessary procedure.

MichaelEischer

the path is necessary information, making the lookup a necessary procedure.

While I agree with the first part, I only partially agree to the second part. This is also why I called restic ls [...] / a workaround. If you only have the information from the snapshots command, then you're right. However, based on the parameters passed to the backup command, it is possible to exactly tell where in the snapshot the files will end up (although the handling of file name collisions is not obvious).

MichaelEischer · 2023-08-19T17:20:07Z

doc/040_backup.rst

@@ -45,6 +45,17 @@ size of the files and directories in ``~/work`` on the local file system. It
 also tells us that only 1.200 GiB was added to the repository. This means that
 some of the data was duplicate and restic was able to efficiently reduce it.

+We just attached the absolute path ``~/work`` to the backup, so the path


I'd prefer to move this to it's own subsection, for example Paths in a snapshot and then show a backup using a relative and an absolute path each followed with the corresponding output of snapshots and ls as illustration.

The jump from ~/work to /home/user/work may also not be obvious to every user, especially those that are unfamiliar with unix shells.

Please create such a subsection, and you are welcome to copy my text, which I then remove from this PR.
Ideally we can work on this branch, Git allows this.

I agree that ~/ could confuse Windows users, but I just wanted to be consistent with the documentation.

Shall I replace the 13 cases of ~/' with /home/user/`in 040_backup alone, or do you want to do that?

MichaelEischer · 2023-08-19T17:22:52Z

doc/040_backup.rst

@@ -45,6 +45,17 @@ size of the files and directories in ``~/work`` on the local file system. It
 also tells us that only 1.200 GiB was added to the repository. This means that
 some of the data was duplicate and restic was able to efficiently reduce it.

+We just attached the absolute path ``~/work`` to the backup, so the path
+within the repository is ``/home/user/work``, depending on your user name.


Paths only exist within snapshots, not the repository.

This is another argument why snapshots should list the attached path and the "path within the snapshot". And maybe inner path is a shorter name

MichaelEischer · 2023-08-19T17:23:58Z

doc/040_backup.rst

@@ -45,6 +45,17 @@ size of the files and directories in ``~/work`` on the local file system. It
 also tells us that only 1.200 GiB was added to the repository. This means that
 some of the data was duplicate and restic was able to efficiently reduce it.

+We just attached the absolute path ``~/work`` to the backup, so the path


What about a plain We backed up the absolute path ..., so the snapshot contains the full path: followed by the output of ls?

I personally think documentation should contain a path like /home/user/work as example for an absolute path.

MichaelEischer · 2023-08-19T17:26:42Z

doc/040_backup.rst

+If we attach a relative path ``work``, the path within the repository is ``/work``.
+
+For example ``restic backup work`` run from ``/home/user`` crates a snapshot
+with a attached path ``/home/user/work`` that contains the path ``/work``


What about [...] creates a snapshot with the directory /home/user/work stored in the snapshot metadata plus showing the output of snapshots?

As the column in the snapshots command is named Directory that would be consistent.

Should "attached path" be used at all?

MichaelEischer · 2023-08-19T17:28:59Z

doc/050_restore.rst

@@ -156,6 +156,31 @@ e.g.:

    $ restic -r /srv/restic-repo dump --path /production.sql latest production.sql | mysql

+This example assumes you attached an absolute path, which means it coincides with the


The path stored in the snapshot metadata (==attached path) is always absolute. I think you mean you ran a backup using an absolute path

This is clearer - I will change that.

damoclark · 2023-08-20T01:44:28Z

In the longer term, a potential solution to simplify this and many other reported issues is being discussed in PR #4026. Part of the proposed approach relevant here is that all snapshots will use relative paths, much like existing archiving tools such as tar and zip.

marbx · 2023-08-20T14:41:49Z

the path is necessary information, making the lookup a necessary procedure.

While I agree with the first part, I only partially agree to the second part. This is also why I called restic ls [...] / a workaround. If you only have the information from the snapshots command, then you're right. However, based on the parameters passed to the backup command, it is possible to exactly tell where in the snapshot the files will end up (although the handling of file name collisions is not obvious).

I did not remember the parameter, the rules are not documented, and therefore I only had the snapshots command. Until snapshots shows both paths, it is not a workaround

marbx · 2023-08-20T15:32:32Z

In the longer term, a potential solution to simplify this and many other reported issues is being discussed in PR #4026. Part of the proposed approach relevant here is that all snapshots will use relative paths, much like existing archiving tools such as tar and zip.

Thanks for pointing to PR #4026, I need more time to read it completely.

In short: I support the idea of snapshots using relative paths!

dump requires absolute path from within the repo

2c37fb2

Added path discrepancy to backup and listing

d57ef6e

MichaelEischer reviewed Aug 19, 2023

View reviewed changes

Entered feedback from MichaelEischer

1925457

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

dump requires absolute path from within the repo #4443

dump requires absolute path from within the repo #4443

marbx commented Aug 9, 2023 •

edited

MichaelEischer commented Aug 13, 2023

marbx commented Aug 16, 2023

MichaelEischer left a comment

MichaelEischer Aug 19, 2023

marbx Aug 20, 2023 •

edited

MichaelEischer Aug 19, 2023

marbx Aug 20, 2023

MichaelEischer Aug 19, 2023

marbx Aug 20, 2023

MichaelEischer Aug 19, 2023

marbx Aug 20, 2023

MichaelEischer Aug 19, 2023

marbx Aug 20, 2023

marbx Aug 20, 2023

damoclark commented Aug 20, 2023

marbx commented Aug 20, 2023 •

edited

marbx commented Aug 20, 2023

		@@ -156,6 +156,31 @@ e.g.:

		$ restic -r /srv/restic-repo dump --path /production.sql latest production.sql \| mysql

		This example assumes you attached an absolute path, which means it coincides with the

dump requires absolute path from within the repo #4443

Are you sure you want to change the base?

dump requires absolute path from within the repo #4443

Conversation

marbx commented Aug 9, 2023 • edited

What does this PR change? What problem does it solve?

Was the change previously discussed in an issue or on the forum?

MichaelEischer commented Aug 13, 2023

marbx commented Aug 16, 2023

MichaelEischer left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

marbx Aug 20, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

damoclark commented Aug 20, 2023

marbx commented Aug 20, 2023 • edited

marbx commented Aug 20, 2023

marbx commented Aug 9, 2023 •

edited

marbx Aug 20, 2023 •

edited

marbx commented Aug 20, 2023 •

edited