Using source-directory w/o xattr fails

[EleRas]

Output of restic version

restic 0.9.0 compiled with go1.10.2 on linux/amd64

How did you run restic exactly?

restic -r rclone:gdrive:restic --verbose --verbose init

enter password for new repository: 
enter password again: 
created restic repository 86c6846e1f at rclone:gdrive:restic

Please note that knowledge of your password is required to access
the repository. Losing your password means that your data is
irrecoverably lost.

restic -r rclone:gdrive:restic --verbose --verbose backup /mnt/photos

/mnt/photos is a mount:

///photo on /mnt/photos type cifs (rw,relatime,vers=default,cache=strict,username=,domain=,uid=1000,forceuid,gid=100,forcegid,addr=192.168.1.17,file_mode=0775,dir_mode=0775,soft,nounix,serverino,mapposix,noperm,rsize=1048576,wsize=1048576,echo_interval=60,actimeo=1,user=,x-systemd.automount)

This mount does NOT allow xattr.

restic output:

open repository
enter password for repository: 
repository 86c6846e opened successfully, password is correct
lock repository
rclone: 2018/05/22 19:25:20 ERROR : locks: error listing: directory not found
load index files
rclone: 2018/05/22 19:25:23 ERROR : index: error listing: directory not found
rclone: 2018/05/22 19:25:23 ERROR : snapshots: error listing: directory not found
start scan on [/mnt/photos]
start backup on [/mnt/photos]
error: NodeFromFileInfo: Listxattr: xattr.List /mnt/photos : no data available
new       /mnt/, saved in 0.022s (0 B added, 13 B metadata)

Files:           0 new,     0 changed,     0 unmodified
Dirs:            1 new,     0 changed,     0 unmodified
Data Blobs:      0 new
Tree Blobs:      2 new
Added:      377 B

processed 0 files, 0 B in 0:16
snapshot 99ae013a saved
context canceled B, total 32856 files 13.350 GiB, 1 errors

Notice how restic does not process any files, even if it tells it has a total of 32856 files. After this restic quits and no data is in snapshot

What backend/server/service did you use to store the repository?

google drive via rclone

Expected behavior

restic creates a snapshot containing data

Actual behavior

restic creates an empty snapshot which can't be restored:

restic -r rclone:gdrive:restic --verbose --verbose restore 99ae013a --target /tmp/tmporestore

enter password for repository: 
repository 86c6846e opened successfully, password is correct
restoring <Snapshot 99ae013a of [/mnt/photos] at 2018-05-22 19:25:23.762135498 +0200 CEST by <removed>> to /tmp/tmprestore
ignoring error for /mnt: Lchown: lchown /tmp/tmprestore/mnt: operation not permitted
There were 1 errors

restic -r rclone:gdrive:restic --verbose --verbose ls -l latest

enter password for repository: 
repository 86c6846e opened successfully, password is correct
snapshot 99ae013a of [/mnt/photos] at 2018-05-22 19:25:23.762135498 +0200 CEST):
drwxr-xr-x     0     0      0 2018-05-10 23:22:39 /mnt

The source directory has an upper layer containing 158 files & directories:

ls -l /mnt/photos | wc -l
158

Steps to reproduce the behavior

• create a source-directory which does not support extended attributes
• try to create a backup

Do you have any idea what may have caused this?

Based on the errormessage I guess missing extended attributes on the source-directory

Do you have an idea how to solve the issue?

no

Did restic help you or made you happy in any way?

Very happy to find a backup - programm using rclone as backend, Works like a charm on local files.

[cameronleger]

I've produced the relevant section of a debug log for this. I would really appreciate any help towards solving this issue because nearly everything I have to backup is on a CIFS file system. It might be worth noting that recent samba updates have modified a few related things, one in particular is that stat --file-system --format=%T now returns smb2 instead of cifs.

2018/05/31 18:04:55 b2/b2.go:282	b2.(*b2Backend).List	1	returned 0 items
2018/05/31 18:04:55 archiver/archiver.go:663	archiver.resolveRelativeTargets	1	targets before resolving: [/mnt/resources/Images/ChildFolder]
2018/05/31 18:04:55 archiver/archiver.go:684	archiver.resolveRelativeTargets	1	targets after resolving: [/mnt/resources/Images/ChildFolder]
2018/05/31 18:04:55 archiver/tree.go:233	archiver.NewTree	1	targets: [/mnt/resources/Images/ChildFolder]
2018/05/31 18:04:55 archiver/tree.go:252	archiver.NewTree	1	result:
/mnt, root "/", meta "/mnt"
    /resources, root "", meta "/mnt/resources"
        /Images, root "", meta "/mnt/resources/Images"
            /ChildFolder, src "/mnt/resources/Images/ChildFolder"

2018/05/31 18:04:55 archiver/file_saver.go:75	archiver.NewFileSaver	1	new file saver with 2 file workers and 8 blob workers
2018/05/31 18:04:55 archiver/archiver.go:758	archiver.(*Archiver).Snapshot	1	starting snapshot
2018/05/31 18:04:55 archiver/archiver.go:479	archiver.(*Archiver).SaveTree	1	/ (1 nodes), parent <nil>
2018/05/31 18:04:55 archiver/archiver.go:479	archiver.(*Archiver).SaveTree	1	/mnt (1 nodes), parent <nil>
2018/05/31 18:04:55 archiver/archiver.go:479	archiver.(*Archiver).SaveTree	1	/mnt/resources (1 nodes), parent <nil>
2018/05/31 18:04:55 archiver/archiver.go:479	archiver.(*Archiver).SaveTree	1	/mnt/resources/Images (1 nodes), parent <nil>
2018/05/31 18:04:55 archiver/archiver.go:311	archiver.(*Archiver).Save	1	/mnt/resources/Images/ChildFolder target "/mnt/resources/Images/ChildFolder", previous <nil>
2018/05/31 18:04:55 archiver/archiver.go:391	archiver.(*Archiver).Save	1	  /mnt/resources/Images/ChildFolder dir
2018/05/31 18:04:55 archiver/archiver.go:204	archiver.(*Archiver).SaveDir	1	/mnt/resources/Images/ChildFolder /mnt/resources/Images/ChildFolder
2018/05/31 18:04:55 restic/node.go:653	restic.(*Node).fillExtendedAttributes	1	fillExtendedAttributes(/mnt/resources/Images/ChildFolder) [] Listxattr: xattr.List /mnt/resources/Images/ChildFolder : no data available
2018/05/31 18:04:55 archiver/archiver.go:402	archiver.(*Archiver).Save	1	SaveDir for /mnt/resources/Images/ChildFolder returned error: NodeFromFileInfo: Listxattr: xattr.List /mnt/resources/Images/ChildFolder : no data available
2018/05/31 18:04:55 archiver/archiver.go:147	archiver.(*Archiver).error	1	item /mnt/resources/Images/ChildFolder: error was filtered by handler, before: "NodeFromFileInfo: Listxattr: xattr.List /mnt/resources/Images/ChildFolder : no data available", after: <nil>
2018/05/31 18:04:55 archiver/archiver.go:569	archiver.(*Archiver).SaveTree	1	waiting on 0 nodes
2018/05/31 18:04:55 repository/repository.go:213	repository.(*Repository).SaveAndEncrypt	164	save id ac08ce34 (tree, 13 bytes)
2018/05/31 18:04:55 repository/packer_manager.go:64	repository.(*packerManager).findPacker	164	create new pack
2018/05/31 18:04:55 repository/repository.go:251	repository.(*Repository).SaveAndEncrypt	164	pack is not full enough (45 bytes)
2018/05/31 18:04:55 repository/packer_manager.go:87	repository.(*packerManager).insertPacker	164	1 packers
2018/05/31 18:04:55 archiver/archiver.go:544	archiver.(*Archiver).SaveTree	1	/mnt/resources, saved subtree Tree<0 nodes> as ac08ce34
2018/05/31 18:04:55 archiver/archiver.go:551	archiver.(*Archiver).SaveTree	1	/mnt/resources, dir node data loaded from /mnt/resources/Images
2018/05/31 18:04:55 restic/node.go:653	restic.(*Node).fillExtendedAttributes	1	fillExtendedAttributes(/mnt/resources/Images) [] Listxattr: xattr.List /mnt/resources/Images : no data available
2018/05/31 18:04:55 archiver/archiver.go:771	archiver.(*Archiver).Snapshot	1	saved tree, error: NodeFromFileInfo: Listxattr: xattr.List /mnt/resources/Images : no data available
2018/05/31 18:04:55 archiver/archiver.go:775	archiver.(*Archiver).Snapshot	1	err is NodeFromFileInfo: Listxattr: xattr.List /mnt/resources/Images : no data available, werr is <nil>
2018/05/31 18:04:55 archiver/archiver.go:781	archiver.(*Archiver).Snapshot	1	error while saving tree: NodeFromFileInfo: Listxattr: xattr.List /mnt/resources/Images : no data available
2018/05/31 18:04:55 restic/lock.go:97	main.unlockRepo	1	unlocking repository with lock PID 23119 on zeus by cameron (UID 1000, GID 100)
lock was created at 2018-05-31 18:04:53 (1.298249822s ago)
storage ID 323b0b7f
2018/05/31 18:04:55 cache/backend.go:37	cache.(*Backend).Remove	1	cache Remove(<lock/323b0b7fb6>)
2018/05/31 18:04:55 b2/b2.go:251	b2.(*b2Backend).Remove	1	Remove <lock/323b0b7fb6>

[cameronleger]

I do not believe this issue lies within Restic's control. It seems that recent samba and/or kernel changes have upset all extended attribute based tools. I've confirmed the same message no data available in even the basic attr binary, and I've seen numerous posts about this same issue across programs. I'm not too hip on the details, but technically it's no longer "unsupported" for samba yet it can cause issues with recent versions.

There is a surefire - perhaps only for the meantime - fix to be applied when mounting the network shares. Add the nouser_xattr option to the mount in fstab and remount the drives. This will essentially force the unsupported message, which Restic et al. will kindly ignore.

[EleRas]

Can confirm: mounting with nouser_xattr allows restic to perform the backup

[fd0]

Thanks for providing more details! While restic is unable to read the extended attributes, I think it should report the error and carry on (read the dir, save the files) instead of aborting. That's something we need to fix within restic.

[joolsr]

I had a recent similar error concerning xattr cifs errors where Duplicity reported errors. This was using Manjaro Linux .
Again mounting the file shares nouser_xattr as above did the trick. I have a feeling you're likely to see more of these unfortunately ..

[joolsr]

I had a recent similar error concerning xattr cifs errors where Duplicity reported errors. This was using Manjaro Linux .
Again mounting the file shares nouser_xattr as above did the trick. I have a feeling you're likely to see more of these type of reported issues unfortunately ..

[MichaelEischer]

Is this still an issue? If yes then it should be sufficient to whitelist ENOTDATA when trying to read extended attributes in

restic/internal/restic/node_xattr.go

Line 29 in 5fd3dbc

if err, ok := e.(*xattr.Error); ok && err.Err == syscall.ENOTSUP {

.

[hoyho]

Is this still an issue? If yes then it should be sufficient to whitelist ENOTDATA when trying to read extended attributes in

restic/internal/restic/node_xattr.go

Line 29 in 5fd3dbc

if err, ok := e.(*xattr.Error); ok && err.Err == syscall.ENOTSUP {

.

Yes, still an issue .
When we use samba share as the backup source it will encounter the same problem.
I fix it by ignoring the error ENOTDATA. Every thing looks good so I cherry pick the commit and submit a PR.
Please take a look

[uok]

using nouser_xattr and related GODEBUG from #2659 helped me fix the problems in my script (https://forum.restic.net/t/error-lstat-errno-524-during-backup/3272)

[MichaelEischer]

Closing this as it should be fixed by #3034.