Library scans fail to add new series

[slaszt]

What happened?

I added a PDF file to a library directory in a containing folder meant for a new series. When I go to scan a library in order to add the new series, the scan hangs for a while and then finishes without adding the new documents to the library (there is no new series created).

Conversely, when I add the same PDF file to an existing series folder, the scan adds it to the existing series as normal.

Folder structure:

Books/
├── Existing Series/
│   └── Existing Series.pdf
└── New Series/
    └── New Series.pdf

The main task I was doing prior to encountering this issue was mass deleting series I no longer wanted (from a single library) from disk and then scanning the series that was deleted. This resulted in the series disappearing, which was the desired outcome, but maybe this method introduced some kind of problem?

#2945 also describes a similar issue, might be related. Relevant logs are posted.

What did you expect?

I expected that the scan would automatically create a new series based on the folder structure on disk. It looks like, based on the logs, that Kavita detected a change in the folders but something is forcing the scan to restart every minute or so.

Kavita Version Number - If you don not see your version number listed, please update Kavita and see if your issue still persists.

0.8.1 - Stable

What operating system is Kavita being hosted from?

Docker (Other)

If the issue is being seen on Desktop, what OS are you running where you see the issue?

Linux

If the issue is being seen in the UI, what browsers are you seeing the problem on?

Firefox

If the issue is being seen on Mobile, what OS are you running where you see the issue?

None

If the issue is being seen on the Mobile UI, what browsers are you seeing the problem on?

No response

Relevant log output

[Kavita] [2024-05-08 16:36:37.271 -04:00  48] [Information] API.Services.Tasks.ScannerService [ScannerService] Beginning file scan on Tabletop
[Kavita] [2024-05-08 16:44:10.639 -04:00  76] [Information] API.Services.TaskScheduler Scheduling ScanFolder for /tabletop/Tabletop Information
[Kavita] [2024-05-08 16:44:10.639 -04:00  76] [Debug] API.Services.Tasks.Scanner.LibraryWatcher [LibraryWatcher] ProcessChange completed in 1ms
[Kavita] [2024-05-08 16:44:10.668 -04:00  76] [Information] API.Services.TaskScheduler Skipped scheduling ScanFolder for /tabletop/Tabletop Information as a job already queued
[Kavita] [2024-05-08 16:44:10.669 -04:00  76] [Debug] API.Services.Tasks.Scanner.LibraryWatcher [LibraryWatcher] ProcessChange completed in 0ms
[Kavita] [2024-05-08 16:44:10.676 -04:00  76] [Information] API.Services.TaskScheduler Skipped scheduling ScanFolder for /tabletop/Tabletop Information as a job already queued
[Kavita] [2024-05-08 16:44:10.676 -04:00  76] [Debug] API.Services.Tasks.Scanner.LibraryWatcher [LibraryWatcher] ProcessChange completed in 0ms
[Kavita] [2024-05-08 16:44:10.686 -04:00  76] [Information] API.Services.TaskScheduler Skipped scheduling ScanFolder for /tabletop/Tabletop Information as a job already queued
[Kavita] [2024-05-08 16:44:10.686 -04:00  76] [Debug] API.Services.Tasks.Scanner.LibraryWatcher [LibraryWatcher] ProcessChange completed in 0ms
[Kavita] [2024-05-08 16:44:10.696 -04:00  76] [Information] API.Services.TaskScheduler Skipped scheduling ScanFolder for /tabletop/Tabletop Information as a job already queued
[Kavita] [2024-05-08 16:44:10.696 -04:00  76] [Debug] API.Services.Tasks.Scanner.LibraryWatcher [LibraryWatcher] ProcessChange completed in 0ms
[Kavita] [2024-05-08 16:44:10.704 -04:00  76] [Information] API.Services.TaskScheduler Skipped scheduling ScanFolder for /tabletop/Tabletop Information as a job already queued
[Kavita] [2024-05-08 16:44:10.704 -04:00  76] [Debug] API.Services.Tasks.Scanner.LibraryWatcher [LibraryWatcher] ProcessChange completed in 0ms
[Kavita] [2024-05-08 16:44:10.705 -04:00  76] [Information] API.Services.TaskScheduler Skipped scheduling ScanFolder for /tabletop/Tabletop Information as a job already queued
[Kavita] [2024-05-08 16:44:10.705 -04:00  76] [Debug] API.Services.Tasks.Scanner.LibraryWatcher [LibraryWatcher] ProcessChange completed in 0ms
[Kavita] [2024-05-08 16:44:10.706 -04:00  76] [Information] API.Services.TaskScheduler Skipped scheduling ScanFolder for /tabletop/Tabletop Information as a job already queued
[Kavita] [2024-05-08 16:44:10.706 -04:00  76] [Debug] API.Services.Tasks.Scanner.LibraryWatcher [LibraryWatcher] ProcessChange completed in 0ms

[Kavita] [2024-05-08 16:46:05.843 -04:00  60] [Information] API.Services.TaskScheduler Enqueuing library scan for: 5
[Kavita] [2024-05-08 16:46:05.844 -04:00  33] [Information] API.Services.Tasks.CleanupService Performing cleanup of Cache directories
[Kavita] [2024-05-08 16:46:05.844 -04:00  33] [Information] API.Services.Tasks.CleanupService Cache directory purged
[Kavita] [2024-05-08 16:46:05.845 -04:00  80] [Information] API.Services.Tasks.ScannerService [ScannerService] Beginning file scan on Tabletop
[Kavita] [2024-05-08 16:46:52.279 -04:00  73] [Information] API.Services.Tasks.ScannerService [ScannerService] Beginning file scan on Tabletop
[Kavita] [2024-05-08 16:47:22.280 -04:00  51] [Information] API.Services.Tasks.ScannerService [ScannerService] Beginning file scan on Tabletop
[Kavita] [2024-05-08 16:48:22.281 -04:00  63] [Information] API.Services.Tasks.ScannerService [ScannerService] Beginning file scan on Tabletop
[Kavita] [2024-05-08 16:49:00.484 -04:00  45] [Information] Serilog.AspNetCore.RequestLoggingMiddleware HTTP GET / responded 200 in 0.6199 ms
[Kavita] [2024-05-08 16:49:11.277 -04:00  51] [Information] Serilog.AspNetCore.RequestLoggingMiddleware HTTP GET /favicon.ico responded 404 in 0.0633 ms
[Kavita] [2024-05-08 16:49:27.707 -04:00  45] [Information] Serilog.AspNetCore.RequestLoggingMiddleware HTTP GET /kavita/api/users/has-library-access?libraryId=5 responded 200 in 0.9841 ms
[Kavita] [2024-05-08 16:49:27.766 -04:00  45] [Information] Serilog.AspNetCore.RequestLoggingMiddleware HTTP GET /kavita/api/series/metadata?seriesId=358 responded 200 in 2.2016 ms
[Kavita] [2024-05-08 16:49:27.777 -04:00  45] [Information] Serilog.AspNetCore.RequestLoggingMiddleware HTTP GET /kavita/api/library/type?libraryId=5 responded 200 in 0.7139 ms
[Kavita] [2024-05-08 16:49:27.781 -04:00  45] [Information] Serilog.AspNetCore.RequestLoggingMiddleware HTTP GET /kavita/api/readinglist/lists-for-series?seriesId=358 responded 200 in 0.8801 ms
[Kavita] [2024-05-08 16:49:27.783 -04:00  62] [Information] Serilog.AspNetCore.RequestLoggingMiddleware HTTP GET /kavita/api/want-to-read?seriesId=358 responded 200 in 2.3390 ms
[Kavita] [2024-05-08 16:49:27.791 -04:00  6] [Information] Serilog.AspNetCore.RequestLoggingMiddleware HTTP GET /kavita/api/series/358 responded 200 in 1.5554 ms
[Kavita] [2024-05-08 16:49:27.793 -04:00  60] [Information] Serilog.AspNetCore.RequestLoggingMiddleware HTTP GET /kavita/api/reader/has-progress?seriesId=358 responded 200 in 5.6144 ms
[Kavita] [2024-05-08 16:49:27.793 -04:00  51] [Information] Serilog.AspNetCore.RequestLoggingMiddleware HTTP GET /kavita/api/reader/continue-point?seriesId=358 responded 200 in 10.2113 ms
[Kavita] [2024-05-08 16:49:27.792 -04:00  62] [Information] Serilog.AspNetCore.RequestLoggingMiddleware HTTP GET /kavita/api/series/next-expected?seriesId=358 responded 200 in 5.8582 ms
[Kavita] [2024-05-08 16:49:27.819 -04:00  118] [Information] Serilog.AspNetCore.RequestLoggingMiddleware HTTP GET /kavita/api/metadata/series-detail-plus?seriesId=358&libraryType=2 responded 200 in 0.9553 ms
[Kavita] [2024-05-08 16:49:27.826 -04:00  45] [Information] Serilog.AspNetCore.RequestLoggingMiddleware HTTP GET /kavita/api/series/series-detail?seriesId=358 responded 200 in 8.3857 ms
[Kavita] [2024-05-08 16:49:27.828 -04:00  51] [Information] Serilog.AspNetCore.RequestLoggingMiddleware HTTP GET /kavita/api/series/all-related?seriesId=358 responded 200 in 10.3267 ms
[Kavita] [2024-05-08 16:49:27.845 -04:00  45] [Information] Serilog.AspNetCore.RequestLoggingMiddleware HTTP GET /kavita/api/scrobbling/library-allows-scrobbling?seriesId=358 responded 200 in 0.4169 ms
[Kavita] [2024-05-08 16:49:27.845 -04:00  51] [Information] Serilog.AspNetCore.RequestLoggingMiddleware HTTP GET /kavita/api/scrobbling/has-hold?seriesId=358 responded 200 in 1.2677 ms
[Kavita] [2024-05-08 16:49:27.847 -04:00  118] [Information] Serilog.AspNetCore.RequestLoggingMiddleware HTTP GET /kavita/api/collection/all-series?ownedOnly=false&seriesId=358 responded 200 in 2.5938 ms
[Kavita] [2024-05-08 16:49:27.847 -04:00  6] [Information] Serilog.AspNetCore.RequestLoggingMiddleware HTTP GET /kavita/api/reader/time-left?seriesId=358 responded 200 in 1.6748 ms
[Kavita] [2024-05-08 16:49:27.852 -04:00  45] [Information] Serilog.AspNetCore.RequestLoggingMiddleware HTTP GET /kavita/api/rating/overall?seriesId=358 responded 200 in 0.8663 ms
[Kavita] [2024-05-08 16:49:32.698 -04:00  45] [Information] Serilog.AspNetCore.RequestLoggingMiddleware HTTP GET /kavita/api/users/has-library-access?libraryId=5 responded 200 in 0.9309 ms
[Kavita] [2024-05-08 16:49:32.709 -04:00  6] [Information] Serilog.AspNetCore.RequestLoggingMiddleware HTTP GET /kavita/api/users/has-library-access?libraryId=5 responded 200 in 1.0243 ms
[Kavita] [2024-05-08 16:49:32.751 -04:00  51] [Information] Serilog.AspNetCore.RequestLoggingMiddleware HTTP GET /kavita/api/library/jump-bar?libraryId=5 responded 200 in 1.0849 ms
[Kavita] [2024-05-08 16:49:32.755 -04:00  6] [Information] Serilog.AspNetCore.RequestLoggingMiddleware HTTP POST /kavita/api/series/v2 responded 200 in 5.6994 ms
[Kavita] [2024-05-08 16:49:32.760 -04:00  51] [Information] Serilog.AspNetCore.RequestLoggingMiddleware HTTP GET /kavita/api/filter responded 200 in 1.8557 ms
[Kavita] [2024-05-08 16:50:22.283 -04:00  62] [Information] API.Services.Tasks.ScannerService [ScannerService] Beginning file scan on Tabletop
[Kavita] [2024-05-08 16:51:07.285 -04:00  62] [Information] API.Services.Tasks.ScannerService [ScannerService] Beginning file scan on Tabletop
[Kavita] [2024-05-08 16:52:07.286 -04:00  80] [Information] API.Services.Tasks.ScannerService [ScannerService] Beginning file scan on Tabletop
[Kavita] [2024-05-08 16:53:07.290 -04:00  6] [Information] API.Services.Tasks.ScannerService [ScannerService] Beginning file scan on Tabletop

Additional Notes

I am running the v0.8.1.0 Docker image from the GitHub container registry.

[slaszt]

So I think that deleting about 50 or so series from disk all at once and/or clicking "Scan series" instead of "Delete" in the UI after deleting from disk is at least partially to blame for this issue.

I have now got it to successfully complete a scan and create the new series that was not appearing earlier. Here are the steps I took:

1. Reverted to a previous version of the database from the backup directory
2. Compared the series visible in the Kavita UI to the series on disk
3. Through the UI, deleted the "phantom series" whose files were deleted from disk
4. Forced a scan of the affected library

Weird problem, but I guess that's what I get for doing it the quicker way (1 fewer click to scan than delete, I thought they would produce the same result). Hopefully you can reproduce it to see what's going on.

[majora2007]

Thanks for the update, will be looking into these scan issues that are being reported soon.