Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

reactively update the library + exif data #792

Draft
wants to merge 13 commits into
base: master
Choose a base branch
from

Conversation

amit-handa
Copy link

@amit-handa amit-handa commented Jan 23, 2023

thank you to all the contibutors/authors for maintaining the repo in a clean and easy to understand, organized way.

note: I am not even sure if this PR is the right way to contribute or begin. I just wanted to address a issue where photoview scan is resource intensive (cant find it rn) and fsnotify is a way to reactively sync photoview .

of course, only fsnotify is not going to ensure it remains in sync. periodic scan is needed to ensure missed events are reprocessed. For the same, I propose comparing media.updated_at db timefield with mediafile's modtime field and reprocess media if latter is updated.

Detail

The server subscribes to fsnotify events for all the albums in the albums table. for each event, it ll sync photoview db + media_cache, as follows:

  • create, write: add/update the media
  • remove: remove the media

if the media file modtime is later than what exists in the db (media.updated_at time), then ideally the corresponding photoview data should be refreshed.

Current status of the PR: it worked on my manual testcase

  • add a photo to the album: photoview detects it and adds it into its collection
  • remove a photo from the album: photoview detects it and removes it from its collection
  • update the photo modtime (touch command) -> press scan: photoview rescans only the media whose modtime > the updated_at field time in the corresponding media row. updates the exifdata (yes, ideally, the operation should be idempotent and other stuff such as media_cache should be idempotently refreshed ... todo)

I ll continue to work on it, test and ask for comments once its done. meanwhile, all advice, knowledge is welcome. golang is not my programming lang of expertise.
note: The PR is in a pretty bad condition currently. ll update it once I feel the code is production grade + testcases are there sufficiently.

@amit-handa amit-handa marked this pull request as draft January 23, 2023 05:20
@amit-handa amit-handa changed the title Fsnotify reactively update the library + exif data Jan 23, 2023
@codecov
Copy link

codecov bot commented Jan 23, 2023

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 70.01%. Comparing base (92bcba0) to head (880ef19).
Report is 45 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #792      +/-   ##
==========================================
+ Coverage   65.51%   70.01%   +4.50%     
==========================================
  Files         137      100      -37     
  Lines       12903    11187    -1716     
  Branches      533      533              
==========================================
- Hits         8453     7833     -620     
+ Misses       4295     3354     -941     
+ Partials      155        0     -155     
Flag Coverage Δ
api ?
ui 70.01% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@amit-handa
Copy link
Author

this PR addresses #201

@norton-chris
Copy link

I'm all for this pull request. Sadly I don't know golang, but I'd like to voice my support for this. This would be super useful for me since I sync photos from my phone, I'd like to see them in Photoview right away (just like Google Photos does). I have a lot of users and photos on my server so periodic scans take a while. Plus this would be better than periodic scans because it scans when files are added, not timer-based.

@kkovaletp
Copy link
Contributor

@amit-handa I created the #910 PR as an alternative to the master and am reviewing open PRs to merge those ready and bring value to the project.
I'm not a developer and don't know GO, but I see that this branch has conflicts and it is still in draft state, so please finish the work and solve conflicts, then let me know

@kkovaletp kkovaletp added help wanted Extra attention is needed pending response A issue or PR which is waiting for clarifications from OP labels Apr 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Extra attention is needed pending response A issue or PR which is waiting for clarifications from OP
Projects
None yet
Development

Successfully merging this pull request may close these issues.

consider inotify as a replacement for manual/polling scans of folders
3 participants