Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Re-checking torrents failing and torrents downloading again / Corrupting files #7982

Open
ned-martin opened this issue Dec 5, 2017 · 13 comments
Labels
Libtorrent Temp folder Waiting upstream Waiting for changes in dependent libraries

Comments

@ned-martin
Copy link

ned-martin commented Dec 5, 2017

qBittorrent version and Operating System

v4.0.2 (64bit) / Windows 8.1 6.3.9600

What is the problem

Checking completed torrent results in large amounts of the torrent being found to be incomplete. Sometimes re-checking will then find it to be complete, but re-checking again is just as likely to find it incomplete again.

This process is corrupting already downloaded torrents if a re-check is run, or if re-check on completion is set, it goes into a loop and continues to corrupt the download.

What is the expected behavior

A completed torrent should check as complete and not get corrupted.

Steps to reproduce

Re-check completed torrents.
Torrent checking stops at (for example) 58%.
Torrent begins to download again from 58%.
re-check torrent again.
Torrent check now completes to 100%
Torrent is moved to completed folder
Torrent re-checks automatically (I have that option enabled)
Re-check stops at 52% (or some other random value, but usually close to the previous value).
Torrent is moved back to incomplete folder
Torrent starts downloading
< process repeats >

Extra info(if any)

The point at which the rechecking fails seems to be random
The files which are marked as incomplete seem to be random (but seems the first file in a folder is more likely to fail)
It may only happen on Torrents where some files were marked to not download
Completed torrents are set to move to a different folder.
Re-check torrent when complete is enabled.
Some torrents were started with earlier versions of qBittorrent and qBittorrent has been upgraded since. I can't confirm if all problematic torrents were initially started with earlier versions.
Torrents are downloaded to a network share
The drive the network share is on is deduplicated. As such, size of file and size on disk are unrelated. This should not matter and I think is hidden via SMB anyway but seemed worth mentioning.

@ned-martin ned-martin changed the title Re-checking torrents failing and torrents downloading again Re-checking torrents failing and torrents downloading again / Corrupting files Dec 5, 2017
@mel-moth
Copy link

mel-moth commented Dec 5, 2017

every single one of my 'finished' torrents that had been completed but marked errored started to redownload. 4000 torrents in my queue after upgrading. There were 20 before. Please god have mercy on me.

when i started to delete the new-old downloads and rebooted everything started downloading again. i'm not long for this world

@ned-martin
Copy link
Author

Yeah... this bug is actually corrupting data. If anyone has not copied their completed torrents elsewhere, they will lose potentially huge amounts of data from this. Fortunately I had already copied most of the data I needed elsewhere, but this has corrupted / deleted a reasonable amount of data, including from some torrents which no longer have seeds, which it is now busy trying to download

I think that probably makes it a high priority.

I still can't replicate it in an way I can explain... but a few things I've noticed:

  • The various paths used by the system have different amounts of slashes on the end, sometimes it's two... which causes errors
  • It appears to have something to do with paths, and/or auto-moving when complete/storing incomplete torrents in a different folder
  • I managed to "fix" (stop them from always failing rechecking) some torrents by various combinations of manually setting their location / turning on or off "store incomplete torrents in..." but unfortunately can't tell you exactly what fixed them, and others are not yet fixed

@zpoison
Copy link

zpoison commented Dec 14, 2017

Same here... I have like 60+ torrent rechecking at every startup for no reason
I delete them from QBT as soon as they are finished so this must prevent them from bugging again

But sometimes I suspect them to reappear back in QBT and so I DL them again without noticing, not sure (sometimes yes sometimes no)

@arctic-carbide
Copy link

I'm encountering the same issue on client v4.5.4 right now. The files are being downloaded to an internal HDD.

Of the torrents I have setup, my copy of the GDC 2017 torrent has been consistently failing on startup saying the files are missing, rechecking, downloading, rechecking again, downloading again, and so on. After multiple failures, I recognized some of the folder names were consistently coming back incomplete. The strange thing is these folders would either have all the files already, have some of the files, or have incomplete files that couldn't play past a certain point. The other torrents of GDC stuff are not showing signs of the same behavior.

The screenshot below shows which folders on my end the client keeps attempting to download and later rechecking and downloading again.

image

@glassez
Copy link
Member

glassez commented Jun 29, 2023

my copy of the GDC 2017 torrent has been consistently failing on startup saying the files are missing, rechecking, downloading, rechecking again, downloading again, and so on.

What do you mean exactly?
First of all is that torrent in "missing files" state should do nothing until you perform some action on it (e.g. resume or re-check).
Does downloading, rechecking again mean that you have "re-check on completion" enabled?
Also note, that providing content of qBittorrent Log is required.

@arctic-carbide
Copy link

arctic-carbide commented Jun 29, 2023

First of all is that torrent in "missing files" state should do nothing until you perform some action on it (e.g. resume or re-check).

The torrent stays in missing file state until I do something. It only rechecks when I reset its location and force recheck. If I attempt to simply resume it without doing that, it throws an error that it can't restore the torrent:

6/29/2023 10:57 AM - Failed to restore torrent. Files were probably moved or storage isn't accessible. Torrent: "Sonniss.com - GDC 2017 - Game Audio Bundle". Reason: "Sonniss.com - GDC 2017 - Game Audio Bundle fast resume rejected. file_stat(E:\Users\arctic\Music\SFX\Sonniss.com - GDC 2017 - Game Audio Bundle\Pole Position - M8 Greyhound Armored Car_\M8_t3_ext_passby_stop_turnaround_away_var_speeds_Schoeps.1.wav): mismatching file size"

Does downloading, rechecking again mean that you have "re-check on completion" enabled?

Yes. After forcing recheck and the torrent finishes rechecking, it then stalls at around 96% and then starts redownloading files from there. I have "re-check on completion" enabled, so it rechecks the files again when the download finishes; but it determines that the files are still missing, stalls again at around 96%, and then tries to redownload those missing files. This cycle continues indefinitely for the same folders it thinks files are missing from.

Previously, I didn't have rechecks on completion enabled. In that case, it would say it was complete and seed normally, which is strange if it wasn't actually complete and contained partial/corrupted files. It wouldn't be until I started qBittorrent again that it would again say files are missing, and so I enabled rechecks to see if this was a recurring problem.

Also note, that providing content of qBittorrent Log is required.

Other than the log line above, qBittorrent didn't capture much of note. The execution log isn't throwing other errors or providing more insight into what's happening, and the appdata log doesn't show anything for the days I had experienced the problem.

@glassez
Copy link
Member

glassez commented Jun 29, 2023

Previously, I didn't have rechecks on completion enabled. In that case, it would say it was complete and seed normally, which is strange if it wasn't actually complete and contained partial/corrupted files.

Could you disable "recheck on completion", let the torrent to complete downloading again and provide a screenshot of qBittorrent window with this torrent selected and General info pane opened?

@arctic-carbide
Copy link

arctic-carbide commented Jun 29, 2023

Could you disable "recheck on completion", let the torrent to complete downloading again and provide a screenshot of qBittorrent window with this torrent selected and General info pane opened?

After disabling "recheck on completion", below are a couple images of the general pane with the torrent selected:
Image - Torrent General Info Pane - After Checking Complete
Image - Torrent General Info Pane - After Download Complete

Some things I've discovered while getting the screenshots:

  1. Affected folders on the torrent (example) with less than 80% progress before downloading begins regularly have duplicate file names in them, but I can only speculate if they're either merged or if only one of them gets downloaded. Of these files, the local file tends to be corrupted at some point in their playback in Windows Media Player, either stopping and throwing an error or having chunks of audio missing.
  2. Affected folders at 80% or higher progress before downloading begins appear to already have downloaded all the files and lack duplicate file names within the torrent. These usually playback completely and it is not clear if they are corrupted at all.
  3. Could be unrelated: the original torrent file has a slightly larger size property (about a 20 byte difference) than the exported torrent after a download completes, but both torrent files have an equal "size on disk" property.

@glassez
Copy link
Member

glassez commented Jun 30, 2023

Affected folders on the torrent (example) with less than 80% progress before downloading begins regularly have duplicate file names in them

I believe it's the cause of the problem. Need to investigate it a bit...

@glassez
Copy link
Member

glassez commented Jun 30, 2023

arvidn/libtorrent#7463

@glassez
Copy link
Member

glassez commented Jul 5, 2023

There are actually two folders, but somehow they merge and files with the same name end up in the same folder.

IIRC, libtorrent replaces invalid characters with underscore.

@Tynqr
Copy link

Tynqr commented Sep 17, 2023

Having the same issue as @arctic-carbide with the same files in the same torrent, for what it's worth. qbittorrent-nox 4.5.2, Debian.

IMG_20230917_153103_752.png

(W) 2023-09-16T17:37:37 - Failed to restore torrent. Files were probably moved or storage isn't accessible. Torrent: "Sonniss.com - GDC 2017 - Game Audio Bundle". Reason: "Sonniss.com - GDC 2017 - Game Audio Bundle fast resume rejected. file_stat(/media/seeds/Sonniss.com - GDC 2017 - Game Audio Bundle/Pole Position - M8 Greyhound Armored Car_/M8_t3_ext_passby_stop_turnaround_away_var_speeds_Schoeps.1.wav): mismatching file size" (W) 2023-09-17T13:27:21 - Failed to restore torrent. Files were probably moved or storage isn't accessible. Torrent: "Sonniss.com - GDC 2017 - Game Audio Bundle". Reason: "Sonniss.com - GDC 2017 - Game Audio Bundle fast resume rejected. file_stat(/media/seeds/Sonniss.com - GDC 2017 - Game Audio Bundle/Pole Position - M8 Greyhound Armored Car_/M8_t3_ext_passby_stop_turnaround_away_var_speeds_Schoeps.1.wav): mismatching file size" (W) 2023-09-17T13:27:57 - Failed to restore torrent. Files were probably moved or storage isn't accessible. Torrent: "Sonniss.com - GDC 2017 - Game Audio Bundle". Reason: "Sonniss.com - GDC 2017 - Game Audio Bundle fast resume rejected. file_stat(/media/seeds/Sonniss.com - GDC 2017 - Game Audio Bundle/Pole Position - M8 Greyhound Armored Car_/M8_t3_ext_passby_stop_turnaround_away_var_speeds_Schoeps.1.wav): mismatching file size" (W) 2023-09-17T13:56:26 - Failed to restore torrent. Files were probably moved or storage isn't accessible. Torrent: "Sonniss.com - GDC 2017 - Game Audio Bundle". Reason: "Sonniss.com - GDC 2017 - Game Audio Bundle fast resume rejected. file_stat(/media/seeds/Sonniss.com - GDC 2017 - Game Audio Bundle/Pole Position - Wellcraft Scarab 34_/Scarab_34_t2_ext_startup_approach_away_shutdown_shotgun_M.1.wav): mismatching file size"

(W) 2023-09-16T04:37:23 - File error alert. Torrent: "Sonniss.com - GDC 2017 - Game Audio Bundle". File: "/media/seeds/Sonniss.com - GDC 2017 - Game Audio Bundle/SoundHolder - Renault Master III dCi135_/Renault Master F3500 dCi135 Exterior Passby High RPM High Speed StereoM10.wav". Reason: "Sonniss.com - GDC 2017 - Game Audio Bundle file_read (/media/seeds/Sonniss.com - GDC 2017 - Game Audio Bundle/SoundHolder - Renault Master III dCi135_/Renault Master F3500 dCi135 Exterior Passby High RPM High Speed StereoM10.wav) error: End of file" (W) 2023-09-16T04:43:58 - File error alert. Torrent: "Sonniss.com - GDC 2017 - Game Audio Bundle". File: "/media/seeds/Sonniss.com - GDC 2017 - Game Audio Bundle/Pole Position - Wellcraft Scarab 34_/Scarab_34_t2_ext_startup_approach_away_shutdown_shotgun_M.wav". Reason: "Sonniss.com - GDC 2017 - Game Audio Bundle file_read (/media/seeds/Sonniss.com - GDC 2017 - Game Audio Bundle/Pole Position - Wellcraft Scarab 34_/Scarab_34_t2_ext_startup_approach_away_shutdown_shotgun_M.wav) error: End of file" (W) 2023-09-16T05:11:21 - File error alert. Torrent: "Sonniss.com - GDC 2017 - Game Audio Bundle". File: "/media/seeds/Sonniss.com - GDC 2017 - Game Audio Bundle/SoundHolder - Renault Master III dCi135_/Renault Master F3500 dCi135 Exterior Passby High RPM High Speed StereoM10.wav". Reason: "Sonniss.com - GDC 2017 - Game Audio Bundle file_read (/media/seeds/Sonniss.com - GDC 2017 - Game Audio Bundle/SoundHolder - Renault Master III dCi135_/Renault Master F3500 dCi135 Exterior Passby High RPM High Speed StereoM10.wav) error: End of file" (W) 2023-09-16T15:38:49 - File error alert. Torrent: "Sonniss.com - GDC 2017 - Game Audio Bundle". File: "/media/seeds/Sonniss.com - GDC 2017 - Game Audio Bundle/SoundHolder - Renault Master III dCi135_/Renault Master F3500 dCi135 Exterior Passby High RPM High Speed StereoM10.wav". Reason: "Sonniss.com - GDC 2017 - Game Audio Bundle file_read (/media/seeds/Sonniss.com - GDC 2017 - Game Audio Bundle/SoundHolder - Renault Master III dCi135_/Renault Master F3500 dCi135 Exterior Passby High RPM High Speed StereoM10.wav) error: End of file"

@luzpaz luzpaz added Libtorrent Waiting upstream Waiting for changes in dependent libraries labels Sep 18, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Libtorrent Temp folder Waiting upstream Waiting for changes in dependent libraries
Projects
None yet
Development

No branches or pull requests

9 participants