Ignore dot files when using non-AppleFS on Mac

[palijn]

I've just ran GPTH on a Takeout tree that was uncompressed and managed on MacOS, on an exFAT filesystem (this is an important detail).

It failed with

Moving photos to output folder : ........................................ 1/26501  Unhandled exception:
PathNotFoundException: Cannot rename file to 'Takeout-GPTH/ALL_PHOTOS/date-unknown/._2012-12-19 09.34.20.jpg', path = 'Takeout/Google Photos/Photos from 2012/._2012-12-19 09.34.20.jpg' (OS Error: No such file or directory, errno = 2)
#0      _checkForErrorResponse (dart:io/common.dart:42)
#1      _File.rename.<anonymous closure> (dart:io/file_impl.dart:318)
<asynchronous suspension>
#2      moveFiles.moveFile (package:gpth/moving.dart:132)
<asynchronous suspension>
#3      moveFiles (package:gpth/moving.dart:137)
<asynchronous suspension>
#4      main.<anonymous closure> (file:///Users/runner/work/GooglePhotosTakeoutHelper/GooglePhotosTakeoutHelper/bin/gpth.dart:365)
<asynchronous suspension>

Actually there have been lots of errors of the kind of Can't get date on Takeout/Google Photos/Sans titre(40)/._IMG-75ca9ab6bc90579b54ac9b0f052041b6-V.jpg
They are resource forks in the AppleDouble format. I don't think those resources would even be useful on anything but MacOS, and anyway they can be recreated at will (nowadays).
Maybe GPTH can simply ignore them ?

The usual workaround is to run dot_clean Takeout , which I am currently running and should fix the issue.

[TheLastGimbus]

hmm, maybe i can just add

if(Platform.isMac) {
  Process.run('dot_clean -m $inputFolder');
}

at the beginning 👀 ? - is it okay to do it on mac (won't break anything etc)

[palijn]

In my case it was ok.
Please note that :
1- it might take a long time without progress information on large file trees
2- the dot files might reappear in the middle of GPTH processing if the user does things like casually browsing the folder contents, so it's not foolproof

I would think that there is no legitimate reason for GPTH to try processing a file starting with dot, so it might be better to ignore them rather than try suppressing them ?

Note again, those files won't even exist if the disk is using a native Apple filesystem. They will would pop up in existence if the Takeout is done on another filesystem like exFAT, or through CIFS/SMB. (Like, uncompressing Takeout over the network and processing gpth on the server.)

[TheLastGimbus]

ehh, i will just pin this for now and see if anyone has such problem again...

[Pancham97]

Hi, didn't take long, but yes, I am facing the same issue with the same Ex-FAT FS. But dot_clean helped me out with that.

[ktang16]

Ran into this issue today, but I unfortunately don't have the zip folders to restart the process. Maybe the readme should mention that you should keep the zip files in case the process fails?

[TheLastGimbus]

yep, done it 👌

[Spageddie28]

Hey, code noob here,

Where do I put the "dot_clean" thing?

[Pancham97]

Hey, code noob here,

Where do I put the "dot_clean" thing?

Hey @Spageddie28 I wrote a blog about the same. Hope this helps: https://panchamkhaitan.com/blog/my-rant-on-google-takeout-for-photos

[TheLastGimbus]

blog

oh, nice!

821533e

😭 i will add this in cmd text okay 😭

"Modified date" of all files. If you recall uploading files to iCloud, you would know that it likes to store in order of "Creation date".

ehh, right. It's time to migrate to other programming lanugage... again.. (since Dart doesn't support this)

But you could do this easier with exiftool

anyway, a nice blog. I hope that maybe in aaaaa..... year? I will solve all those issues with gpth too :DDD

[Pancham97]

Hey @TheLastGimbus, did you try running the Terminal command I have mentioned in the blog? Did that not help with the creation date?

[TheLastGimbus]

the for f in *.[iI][mM][gG]; do m=.... ? do you suggest i could use it as a sub-command in gpth itself? Mayyybe, but it's an OS/shell dependent thing (i would need 3 versions) that's very easy to break and mess up

[Pancham97]

the for f in *.[iI][mM][gG]; do m=.... ? do you suggest i could use it as a sub-command in gpth itself? Mayyybe, but it's an OS/shell dependent thing (i would need 3 versions) that's very easy to break and mess up

I can understand. By the way, great work with this tool, @TheLastGimbus! If you're working on something, I'd love to collaborate with you on that.

[jestlin15]

Hi @Pancham97 , thank you for the informative blog. Can I please ask, when do I run the for f in *.[iI][mM][gG]; do m="$(stat -f'%Sm' -t "%m/%d/%Y %H:%M:%S" "$f")"; SetFile -m "$m" -d "$m" "$f"; done code? After I did the steps for by Gpth or before?

[Pancham97]

Hi @Pancham97 , thank you for the informative blog. Can I please ask, when do I run the for f in *.[iI][mM][gG]; do m="$(stat -f'%Sm' -t "%m/%d/%Y %H:%M:%S" "$f")"; SetFile -m "$m" -d "$m" "$f"; done code? After I did the steps for by Gpth or before?

Hey @jestlin15, I maybe mistaken given that I wrote this blog a while ago, but I think I didn't use gpth for the second try. I went with another tool. It basically does the same thing as gpth. So, maybe, you need to try the for f in *... after gpth operations have exited successfully.

[zhaotianjing]

I have the same issue and I'm using Mac.

Error message is:

Finding albums (this may take some time, dont worry :) ...
Moving photos to output folder : ........................................ 1/45649 Unhandled exception:
PathNotFoundException: Cannot rename file to '/Volumes/SeagateExp/output/ALL_PHOTOS/._20191128_172905(1).jpg', path = '/Volumes/SeagateExp/Takeout/Takeout 2/Google Photos/Photos from 2019/._20191128_172905.jpg' (OS Error: No such file or directory, errno = 2)

I don't have a file named ._20191128_172905.jpg but I have 20191128_172905.jpg

[TheLastGimbus]

Well do you have "show hidden files" enabled?

Anyway, apple may go as far as not showing those files at all in their own viewer but they are there... Or something 🤷