Understanding --patterns-from and root paths (aka recursion roots aka roots)

[blueray453]

My patterns.lst file is located in /media/ismail/WDPurple6TB/8TBRaid0_Backup/patterns.lst. The file looks like:

P sh
R /media/ismail/8TBRaid0/

! _Backup/RSYNC
! _Download
! _Movies
! _QEMU
! .Trash-1000
! lost+found

I have a command like:

$ borg create --patterns-from patterns.lst --dry-run /media/ismail/WDPurple6TB/8TBRaid0_Backup::{hostname}-{now:%Y-%m-%d_%H:%M:%S} /media/ismail/8TBRaid0

It is giving error:

/media/ismail/8TBRaid0/lost+found: dir_open: [Errno 13] Permission denied: 'lost+found'

I am not understanding what is going on here.

• Did I put patterns.lst in the right location?
• If I want to put patterns.lst in /media/ismail/WDPurple6TB/ then what would the relative path look like?
• why the command is looking at lost+found though i have excluded it?

[blueray453]

One more thing I know I can use --patterns-from with borg recreate.

So, if I run borg recreate --patterns-from patterns.lst /media/ismail/WDPurple6TB/8TBRaid0_Backup it will delete newly excluded files. Is it a correct statement?

The idea is, I want to add following two lines in a script so that if I ignore a directory (!) which i previously added, it will completely delete it and recover space.

borg recreate --patterns-from patterns.lst /media/ismail/WDPurple6TB/8TBRaid0_Backup
borg compact /media/ismail/WDPurple6TB/8TBRaid0_Backup

[ThomasWaldmann]

There is no "default location" for a patterns file, the path you give to --patterns-from will be just opened as it is.

Likely your patterns are not correctly matching. Read the docs about patterns and be aware that the include/exclude patterns are not relative to any of the roots you give.

You can and should tests your patterns for create or recreate by using --dry-run --list before running the command without --dry-run.

[blueray453]

Do you mean i have to give absolute path to --patterns-from. In this particular case, what should i put infront of --patterns-from.

What does it mean to be a root path.

The Documentation says:

Using these, you may specify the backup roots (starting points) and patterns for inclusion/exclusion. A root path starts with the prefix R, followed by a path (a plain path, not a file pattern). An include rule starts with the prefix +, an exclude rule starts with the prefix -, an exclude-norecurse rule starts with !, all followed by a pattern.

So, what I understood is that the root path is the common part of the beginning of the path.

I want to mention that the following works

R /

! media/ismail/8TBRaid0/lost+found
! media/ismail/8TBRaid0/_Backup/RSYNC
! media/ismail/8TBRaid0/_Download
! media/ismail/8TBRaid0/_Movies
! media/ismail/8TBRaid0/_QEMU
! media/ismail/8TBRaid0/.Trash-1000
! media/ismail/8TBRaid0/lost+found

The following also works:

P sh
R /

! /media/ismail/8TBRaid0/lost+found
! /media/ismail/8TBRaid0/_Backup/RSYNC
! /media/ismail/8TBRaid0/_Download
! /media/ismail/8TBRaid0/_Movies
! /media/ismail/8TBRaid0/_QEMU
! /media/ismail/8TBRaid0/.Trash-1000
! /media/ismail/8TBRaid0/lost+found

The following also works:

P sh
R /media/ismail/8TBRaid0/

! media/ismail/8TBRaid0/lost+found
! media/ismail/8TBRaid0/_Backup/RSYNC
! media/ismail/8TBRaid0/_Download
! media/ismail/8TBRaid0/_Movies
! media/ismail/8TBRaid0/_QEMU
! media/ismail/8TBRaid0/.Trash-1000
! media/ismail/8TBRaid0/lost+found

So, my point is, i am not understating what a root path is and what is its purpose? This is because as long as I give absolute path to the patterns, the root path does not matter.

[blueray453]

Just to be more clear, the documentation says:

backup roots (starting points)

I want to know starting points of what exactly?

[ThomasWaldmann]

The root paths (aka recursion roots aka roots) are the starting points of the recursion into the filesystem.

E.g. if you call borg create repo::archive ./.ssh /etc, there are 2 roots:

• ./.ssh which is a relative path. relative to the current working directory.
• /etc which is a absolute path.

So, the recursion will start from each of these and (by default) back up everything in there.

The (normalized) paths it will encounter might look like:

.ssh
.ssh/known_hosts
...
etc
etc/fstab
etc/hosts
...
etc/apt/sources.list
...

Note that there is no ./ in front of .ssh and there is no / in front of etc.

These paths will be fed into the matcher which then matches it against your list of patterns (so you can exclude/include stuff).

The first matching pattern will then decide whether to include/exclude the file.

[blueray453]

I think I understood everything.

Here is my command

borg create --patterns-from /media/ismail/WDPurple6TB/patterns.lst --dry-run /media/ismail/WDPurple6TB/8TBRaid0_Backup::{hostname}-{now:%Y-%m-%d_%H:%M:%S}

Here is my patterns.lst file:

P sh
R /media/ismail/8TBRaid0

! media/ismail/8TBRaid0/lost+found
! media/ismail/8TBRaid0/_Backup/RSYNC
! media/ismail/8TBRaid0/_Download
! media/ismail/8TBRaid0/_Movies
! media/ismail/8TBRaid0/_QEMU
! media/ismail/8TBRaid0/.Trash-1000
! media/ismail/8TBRaid0/lost+found

As I have already mentioned what to backup (R /media/ismail/8TBRaid0/), I did not add it in the command (I have removed /media/ismail/8TBRaid0 from the end of the command). Please let me know whether my understanding is correct.

[blueray453]

Just to confirm the following also works:

borg recreate --patterns-from patterns.lst /media/ismail/WDPurple6TB/8TBRaid0_Backup
borg compact /media/ismail/WDPurple6TB/8TBRaid0_Backup