Unexpected error Cannot read properties of undefined (reading 'FileModifyDate')

[inhahe]

Bug description

GNU nano 7.2 thumbsup.log
2023-05-31T02:20:38.800Z thumbsup:warn dcraw (https://www.cybercom.net/~dcoffin/dcraw/) is not installed. You will not be able to process RAW photos.
2023-05-31T02:20:41.219Z thumbsup:error TypeError: Cannot read properties of undefined (reading 'FileModifyDate')
2023-05-31T02:20:41.219Z thumbsup:error at Stream. (/thumbsup/node_modules/thumbsup/src/components/index/index.js:85:45)
2023-05-31T02:20:41.219Z thumbsup:error at Stream.emit (node:events:513:28)
2023-05-31T02:20:41.219Z thumbsup:error at stream.write (/thumbsup/node_modules/event-stream/index.js:61:10)
2023-05-31T02:20:41.219Z thumbsup:error at Stream.ondata (node:internal/streams/legacy:20:31)
2023-05-31T02:20:41.219Z thumbsup:error at Stream.emit (node:events:513:28)
2023-05-31T02:20:41.219Z thumbsup:error at Stream.reemit (/thumbsup/node_modules/duplexer/index.js:70:25)
2023-05-31T02:20:41.219Z thumbsup:error at Stream.emit (node:events:513:28)
2023-05-31T02:20:41.219Z thumbsup:error at drain (/thumbsup/node_modules/through/index.js:36:16)
2023-05-31T02:20:41.219Z thumbsup:error at stream.queue.stream.push (/thumbsup/node_modules/through/index.js:45:5)
2023-05-31T02:20:41.219Z thumbsup:error at parser.onValue (/thumbsup/node_modules/JSONStream/index.js:118:16)

Steps to reproduce

I followed the instructions for running the docker image at https://thumbsup.github.io/docs/2-installation/docker/.

sudo docker run -t -v "$(pwd):/home/inhahe/" -u $(id -u):$(id -g) ghcr.io/thumbsup/thumbsup thumbsup --input /home/inhahe/pics --output /home/inhahe/pics2

I noticed that my pics have Modify dates earlier than their Change, Birth and Access dates, I guess as a side effect of copying them from the host (Windows) on VirtualBox. Maybe that's the problem?

update: I just "touch"ed all the pics and also removed a subdirectory of the input directory that I'd previously specified as the output directory, and the error went away. Not sure which thing I did made it go away. But now I have another problem. I'll post another issue for it.

update 2: I just changed my docker command to sudo docker run -t -v "/home/inhahe:/home/inhahe" -u $(id -u):$(id -g) ghcr.io/thumbsup/thumbsup thumbsup --input /home/inhahe/pics --output /home/inhahe/pics2 --log trace so that it actually recognizes all the files now, and now I'm getting this error again.

Here's the full log trace output:

thumbsup:warn dcraw (https://www.cybercom.net/~dcoffin/dcraw/) is not installed. You will not be able to process RAW photos. +0ms
thumbsup:info Indexing folder [started] +0ms
thumbsup:info Differences between disk and index { unchanged: 0, added: 3159, modified: 0, deleted: 0, total: 3159 } +0ms
thumbsup:debug Split files into 1 batches for exiftool +0ms
thumbsup:debug Calling exiftool with 3159 files +1ms
thumbsup:trace Exiftool output: Invalid TAG name: "pihel.jpg"
+0ms
thumbsup:trace Exiftool output: Warning: Tag 'DwYCV2uvfqUDYxYruNHY6_F6UZ0Lc4jaHCg9wPOqPuVdCDrareKrw' is not defined
+363ms
thumbsup:trace Exiftool output: Warning: [minor] Fixed incorrect URI for xmlns:MicrosoftPhoto - 318281_10150681863854556_62848724555_9044650_151792582_n.jpg
+413ms
thumbsup:trace Exiftool output: Warning: [minor] Text/EXIF chunk(s) found after PNG IDAT (may be ignored by some readers) - 5378f1c4cb76e196955306.png
+149ms
thumbsup:trace Exiftool output: Warning: [minor] Fixed incorrect URI for xmlns:MicrosoftPhoto - 549826_3287571981988_1048332263_33092038_1549521788_n.jpg
+15ms
thumbsup:trace Exiftool output: Warning: [minor] Fixed incorrect URI for xmlns:MicrosoftPhoto - 62838_162718547075579_100000123047702_561852_2559810_n.jpg
+152ms
thumbsup:error TypeError: Cannot read properties of undefined (reading 'FileModifyDate')
thumbsup:error     at Stream.<anonymous> (/thumbsup/node_modules/thumbsup/src/components/index/index.js:85:45)
thumbsup:error     at Stream.emit (node:events:513:28)
thumbsup:error     at stream.write (/thumbsup/node_modules/event-stream/index.js:61:10)
thumbsup:error     at Stream.ondata (node:internal/streams/legacy:20:31)
thumbsup:error     at Stream.emit (node:events:513:28)
thumbsup:error     at Stream.reemit (/thumbsup/node_modules/duplexer/index.js:70:25)
thumbsup:error     at Stream.emit (node:events:513:28)
thumbsup:error     at drain (/thumbsup/node_modules/through/index.js:36:16)
thumbsup:error     at stream.queue.stream.push (/thumbsup/node_modules/through/index.js:45:5)
thumbsup:error     at parser.onValue (/thumbsup/node_modules/JSONStream/index.js:118:16) +0ms
thumbsup:error 
thumbsup:error Unexpected error Cannot read properties of undefined (reading 'FileModifyDate') +0ms
thumbsup:error 
thumbsup:error ┌───────────────────────────────────────────────────────────────────────────┐
thumbsup:error │                                                                           │
thumbsup:error │  Something went wrong!                                                    │
thumbsup:error │                                                                           │
thumbsup:error │  An unexpected error occurred and the gallery didn't build successfully.  │
thumbsup:error │  This is most likely an edge-case that hasn't been tested before.         │
thumbsup:error │                                                                           │
thumbsup:error │  Please check the logs at /home/inhahe/pics2/thumbsup.log.                │
thumbsup:error │  To help improve thumbsup and hopefully resolve your problem,             │
thumbsup:error │  you can raise an issue at https://github.com/thumbsup/thumbsup/issues.   │
thumbsup:error │                                                                           │
thumbsup:error └───────────────────────────────────────────────────────────────────────────┘
thumbsup:error  +24ms

[inhahe]

Update: I tried moving half the files in my pics directory and all the subdirectories to a temporary directory, and then running thumbsup on the pics directory. My plan was to keep removing or adding half until I can narrow down what file is doing it. But upon removing half of them and all the subdirectories, I get this:

Gallery generated successfully! │
│ 1 albums, 0 photos, 0 videos

Even though I still have hundreds of pics in the directory.

I tried thumbsup --input pics --output pics2, thumbsup --input ./pics --output ./pics2, thumbsup --input /home/inhahe/pics --output /home/inhahe/pics2, thumbsup --input ./pics/ --output ./pics2/, etc.

ls -all pics doesn't say anything about them being symlinks so I guess they're not symlinks.

[inhahe]

Update: I tried leaving just one file in pics, and I got this error:

2023-05-31T14:05:53.564Z thumbsup:warn gifsicle (http://www.lcdf.org/gifsicle) is not installed. You will not be able to process animated GIFs.
2023-05-31T14:05:54.452Z thumbsup:error Error: Invalid JSON (Unexpected "N" at position 0 in state STOP)
2023-05-31T14:05:54.452Z thumbsup:error     at proto.charError (/home/linuxbrew/.linuxbrew/lib/node_modules/thumbsup/node_modules/jsonparse/jsonparse.js:90:16)
2023-05-31T14:05:54.452Z thumbsup:error     at proto.write (/home/linuxbrew/.linuxbrew/lib/node_modules/thumbsup/node_modules/jsonparse/jsonparse.js:154:23)
2023-05-31T14:05:54.452Z thumbsup:error     at Stream.<anonymous> (/home/linuxbrew/.linuxbrew/lib/node_modules/thumbsup/node_modules/JSONStream/index.js:23:12)
2023-05-31T14:05:54.452Z thumbsup:error     at stream.write (/home/linuxbrew/.linuxbrew/lib/node_modules/thumbsup/node_modules/through/index.js:26:11)
2023-05-31T14:05:54.452Z thumbsup:error     at Transform.ondata (/home/linuxbrew/.linuxbrew/lib/node_modules/thumbsup/node_modules/readable-stream/lib/_stream_readable.js:629:20)
2023-05-31T14:05:54.452Z thumbsup:error     at Transform.emit (node:events:511:28)
2023-05-31T14:05:54.452Z thumbsup:error     at addChunk (/home/linuxbrew/.linuxbrew/lib/node_modules/thumbsup/node_modules/readable-stream/lib/_stream_readable.js:279:12)
2023-05-31T14:05:54.452Z thumbsup:error     at readableAddChunk (/home/linuxbrew/.linuxbrew/lib/node_modules/thumbsup/node_modules/readable-stream/lib/_stream_readable.js:262:11)
2023-05-31T14:05:54.452Z thumbsup:error     at Readable.push (/home/linuxbrew/.linuxbrew/lib/node_modules/thumbsup/node_modules/readable-stream/lib/_stream_readable.js:228:10)
2023-05-31T14:05:54.452Z thumbsup:error     at Transform.push (/home/linuxbrew/.linuxbrew/lib/node_modules/thumbsup/node_modules/readable-stream/lib/_stream_transform.js:132:32)

My only guess is that I'm getting three different failure modes depending on which file that it doesn't like it comes across first. I'm guessing it's because of the filenames, which are coming from Windows. The one pic I left in pics this time has the name #41 Roomba Led Long Exposure.jpg.

[inhahe]

Update: I renamed #41 Roomba Led Long Exposure.jpg to 41 Roomba Led Long Exposure.jpg, and it worked with only that single file in the pics directory.

[inhahe]

So I did detox -r /home/inhahe/pics and that solved the problem. But now I have a new problem. I'll add another issue.

[rprieto]

Hi @inhahe, thanks for your troubleshooting and for reporting this.
I think there might be multiple issues at play here.

Regarding the # character it seems that:

• exiftool will ignore any file paths starting with # (maybe because it thinks it's a comment, or maybe due to the way it parses its input on stdin).
• if this was the only file, then it thinks you didn't pass anything and prints an error message on stdout. Because we normally expect JSON this crashes the parser.

I can reproduce this with:

Command	Description	Result
Normal file	echo 'a.jpg' | exiftool -@ -	OK
File starting with #	echo '#a.jpg' | exiftool -@ -	Error

In any case this is an issue, because even if it's not the only file it would get ignored.
I will look into the right syntax for exiftool to accept them.

Regarding the other issues, were they also solved by running detox?

[inhahe]

Yes, using detox solves my problems.

It seems without using detox, there are at least three different kinds of errors I get as a result of three different possibilities in filenames. One is spaces, I forget what the other ones are (I guess one was the # symbol).

It seems it would be more ideal if it weren't necessary to use detox to fix all the filenames before running thumbsup. And as it is I have to keep copies of all my pic directories that I want to put on the web, and every time I save a new pic in one of those directories I have to save it in two places, because I store my pics on Windows, and every now and then I rsync them to my Linux VM to run thumbsup on them, and I don't want to run detox on my main Windows directories, but if I just rsynced them to Linux where I run detox on the directories, then it would re-add all the filenames that are different from their detoxed names, so I have to have copies of them all in directories on Windows that I detox every time I update my web pics. So that's four copies of my pics in total that I have to store on my PC: my main Windows directories, my detoxed Windows directories for the web, my detoxed Linux directory to use as the source for the thumbsup command (since it doesn't seem to work if I try to run it directly on my mounted Windows directory for some reason), and the destination directory for Thumbsup.