Detect a folder re-appearing more quickly (e.g. on re-mount)

[megabitsenmzq]

What happened?

I'm syncing the root of a removable HDD with another computer. After unmounted the HDD, Syncthing immediately knew there was something wrong and told me "Folder path missing".

But after I put the HDD back, Syncthing couldn't detect there was a change but continued to tell me "Folder path missing". Even though the folder exists now. The only thing I can do is to rescan or set the rescan timer to a short interval.

Syncthing version

v1.27.4

Platform & operating system

M1 MacBook Air macOS Sonoma 14.3.1

Browser version

No response

Relevant log output

No response

[calmh]

I think this is expected. There is no magic to instantly detect the presence of a previously-failed folder, we need to actually look at it which happens in the scheduled scan.

We probably get the removed event "for free" because we're watching the folder for changes and it flags some kind of change. When it's not there any more there is nothing to watch and we don't get any events.

I suppose it would be possible to specifically do something operating system specific to watch for mount events, if there is such a thing, or possibly add a watch on the parent dir and see if we get anything from there, but I don't really see it worth the effort to be honest...

[megabitsenmzq]

Maybe we can have a global setting to set an interval to rescan the folder when something fails. Then we can keep the regular interval intact, but recheck the folder every 5 minutes on fail to take it back. I guess this is easier to do than watching some file system events.

By the way, I just switched from resilio-sync, it can detect it immediately when I put the HDD back. I don't know how it managed to do that.

[megabitsenmzq]

I managed to create a workflow in "Keyboard Maestro" with Syncthing REST API to solve this. But still, I hope we can have this solved by Syncthing itself.

[megabitsenmzq]

I found a bigger problem. It seems that after the HDD disappears and appears again after several hours, Syncthing starts to reindex the whole disk. I remember if it just disappears for a little while, the indexing is very fast, and won’t do it all over again. This HDD needs days to be fully indexed. Do you have any idea about this problem? I’m trying to reproduce it again to confirm.

[imsodin]

I found a bigger problem. It seems that after the HDD disappears and appears again after several hours, Syncthing starts to reindex the whole disk. I remember if it just disappears for a little while, the indexing is very fast, and won’t do it all over again. This HDD needs days to be fully indexed. Do you have any idea about this problem? I’m trying to reproduce it again to confirm.

That clearly shouldn't happen, and I am somewhat confident saying can't happen without external interventation/failure (OS, hardware, user). Lets take debugging of this to the forum please, as that's where we do support (e.g. due to lengthy exchanges that shouldn't cause notifications in github): https://forum.syncthing.net/
Please explain there what exactly you did in the process, what the paths involved are (syncthing folder and mount path), filesystems, logs if you have any, ... - anything that might help.

As for the original request: I am somewhat certain this came up before, but I can't find any issue for it.
Not doing anything smart, just re-checking for "re-appearance" more frequently than the full rescan interval for a folder with a missing path or marker seems like reasonable feature request. Many people have very long rescan intervals, so a remount won't be picked up in a long time. And it's not a big change for syncthing. I don't think we need to make this configurable though, just doing it maybe every minute should both be unobtrusive and fast enough. If a user needs an effect asap after mounting, they can take manual intervention.
I took the liberty to re-title and classify the ticket as such, please let me know if that misrepresents your intent @megabitsenmzq or if you think this isn't a worthwhile thing to do (resp. is flawed/doesn't work) @calmh .

[megabitsenmzq]

Thank you for your reply! The title change is good. Yep, you need to knock the folder more when it's missing. That's exactly what I want. Making it configurable is just optional. I'll post the indexing problem in the forum later today.

[calmh]

I think we maybe have a larger point of possible improvement in the handling of rescan intervals... now we just set the one rescan interval which is used all the time, and we tweak what that interval is depending on wether we expect to use filesystem notifications or not. I think it would be better to have two intervals, one (longer) that is used when the folder is healthy and notifications are online, and a different (shorter) one that is used when the folder is unhealthy or the watcher is offline...

[megabitsenmzq]

I posted https://forum.syncthing.net/t/syncthing-tries-to-index-an-unmounted-volume/21784

(Update: index problem resolved)