Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Failed Archive extraction is recognised, but nothing is done about it #4393

Open
mihawk90 opened this issue Nov 15, 2023 · 0 comments
Open

Failed Archive extraction is recognised, but nothing is done about it #4393

mihawk90 opened this issue Nov 15, 2023 · 0 comments
Assignees
@vuolter
Labels
feature request New enhancement request pyLoad Next

Comments

@mihawk90
Copy link
Contributor

mihawk90 commented Nov 15, 2023 •
edited

Description

When pyload-ng encounters a Bad Archive, due to a broken download or whatever, it logs this, but does nothing about it. Ideally this should restart the download of said archives, or at least mark them as a failed extraction in the package list.*
However, as it stands the files remain as finished (or skipped for mirrored links) in the Package list and in the files, so you'd never know unless you manually check either the log or the files section.

*Note this should probably also have a maximum number of retries so it doesn't keep downloading the same files over and over in case the file is actually corrupted on the hoster side.

Debug log

Just an excerpt of the relevant section:

...
[2023-11-15 19:27:42]  INFO                pyload  ADDON ExtractArchive: DLvK-1080-01x02.part1.rar | Extracting finished
[2023-11-15 19:27:42]  DEBUG               pyload  ADDON ExtractArchive: Extracted files: ['/mnt/media/_unsorted_stuff/_pyload-extracted/Die.Legende.von.Korra.S01/Die.Legende.von.Korra.S01E02.Wie.ein.Blatt.im.Wind.German.DL.1080p.AmazonHD.h264-GDR.mkv']
[2023-11-15 19:27:42]  DEBUG               pyload  ADDON ExternalScripts: No script found under folder `archive_extracted`
[2023-11-15 19:27:42]  INFO                pyload  ADDON ExtractArchive: DLvK-1080-01x03.part1.rar | Extract to: /mnt/media/_unsorted_stuff/_pyload-extracted/Die.Legende.von.Korra.S01
[2023-11-15 19:27:42]  DEBUG               pyload  ADDON ExtractArchive: Password: None
[2023-11-15 19:27:42]  DEBUG               pyload  ADDON ExtractArchive: Verifying using password: None
[2023-11-15 19:27:42]  DEBUG               pyload  EXTRACTOR UnRar: EXECUTE unrar l -scf -o- -or -x*.nfo -x*.DS_Store -xindex.dat -xthumb.db -y -c- -p- -v /downloads/Die.Legende.von.Korra.S01/DLvK-1080-01x03.part1.rar
[2023-11-15 19:27:42]  DEBUG               pyload  EXTRACTOR UnRar: EXECUTE unrar l -scf -o- -or -x*.nfo -x*.DS_Store -xindex.dat -xthumb.db -y -c- -p- -v /downloads/Die.Legende.von.Korra.S01/DLvK-1080-01x03.part1.rar
[2023-11-15 19:27:42]  INFO                pyload  ADDON ExtractArchive: DLvK-1080-01x03.part1.rar | Password protected
[2023-11-15 19:27:42]  DEBUG               pyload  ADDON ExtractArchive: Password was wrong
[2023-11-15 19:27:42]  DEBUG               pyload  ADDON ExtractArchive: Verifying using password: <password>
[2023-11-15 19:27:42]  DEBUG               pyload  EXTRACTOR UnRar: EXECUTE unrar l -scf -o- -or -x*.nfo -x*.DS_Store -xindex.dat -xthumb.db -y -c- -p<password> -v /downloads/Die.Legende.von.Korra.S01/DLvK-1080-01x03.part1.rar
[2023-11-15 19:27:42]  DEBUG               pyload  ADDON ExtractArchive: Password Correct
[2023-11-15 19:27:42]  DEBUG               pyload  ADDON ExtractArchive: Extracting using password:<password>
[2023-11-15 19:27:42]  DEBUG               pyload  EXTRACTOR UnRar: EXECUTE unrar x -scf -o- -or -x*.nfo -x*.DS_Store -xindex.dat -xthumb.db -y -c- -p<password> /downloads/Die.Legende.von.Korra.S01/DLvK-1080-01x03.part1.rar /mnt/media/_unsorted_stuff/_pyload-extracted/Die.Legende.von.Korra.S01
[2023-11-15 19:27:53]  ERROR               pyload  ADDON ExtractArchive: DLvK-1080-01x03.part1.rar | Archive error | ERROR: Bad archive /downloads/Die.Legende.von.Korra.S01/DLvK-1080-01x03.part3.rar
Traceback (most recent call last):
  File "/lsiopy/lib/python3.11/site-packages/pyload/plugins/addons/ExtractArchive.py", line 501, in _extract
    archive.extract(password)
  File "/lsiopy/lib/python3.11/site-packages/pyload/plugins/extractors/UnRar.py", line 213, in extract
    raise ArchiveError(err)
pyload.plugins.base.extractor.ArchiveError: ERROR: Bad archive /downloads/Die.Legende.von.Korra.S01/DLvK-1080-01x03.part3.rar
[2023-11-15 19:27:53]  DEBUG               pyload  ADDON ExternalScripts: No script found under folder `archive_extract_failed`
[2023-11-15 19:27:53]  ERROR               pyload  ADDON ExtractArchive: DLvK-1080-01x03.part1.rar | Extract failed
Traceback (most recent call last):
  File "/lsiopy/lib/python3.11/site-packages/pyload/plugins/addons/ExtractArchive.py", line 330, in extract
    new_files = self._extract(
                ^^^^^^^^^^^^^^
  File "/lsiopy/lib/python3.11/site-packages/pyload/plugins/addons/ExtractArchive.py", line 568, in _extract
    raise Exception(self._("Extract failed"))
Exception: Extract failed
[2023-11-15 19:27:53]  INFO                pyload  ADDON ExtractArchive: DLvK-1080-01x04.part1.rar | Extract to: /mnt/media/_unsorted_stuff/_pyload-extracted/Die.Legende.von.Korra.S01
...

Screenshots

image
Note both are skipped in this case, it's because my password file wasn't accessible so the first extraction attempt failed. I restarted the package and it skipped over every link, then extracted. I don't think this should have any bearing on the behaviour though.

Additional references

From what I can tell from the relevant code:

pyload/src/pyload/plugins/addons/ExtractArchive.py

Lines 560 to 568 in a65a968

except ArchiveError as exc:
self.log_error(name, self._("Archive error"), exc)
except Exception as exc:
self.log_error(name, self._("Unknown error"), exc)
self.m.dispatch_event("archive_extract_failed", pyfile, archive)
raise Exception(self._("Extract failed"))

It seems this is really just rudimentary error handling (and that's fine), hence why I categorised it as a feature request rather then a bug report.

Also note, I know why this file is bad, so that's not the point here. It downloaded some HTML file rather then an actual archive, but I haven't figured out yet what happened here so I had no reason to report it:

$ file DLvK-1080-01x03.part3.rar 
DLvK-1080-01x03.part3.rar: HTML document, ASCII text, with CRLF, LF line terminators

edit:
Seems it recognised it was a bad file but the restart clashed with "Skip existing files":

[2023-11-14 07:17:50]  INFO                pyload  Download restarted: DLvK-1080-01x03.part3.rar | Html file
[2023-11-14 07:17:50]  INFO                pyload  Download starts: DLvK-1080-01x03.part3.rar
[2023-11-14 07:17:50]  DEBUG               pyload  ADDON ExternalScripts: No script found under folder `download_preparing`
[2023-11-14 07:17:50]  DEBUG               pyload  ADDON UserAgentSwitcher: Setting connection timeout to 60 seconds
[2023-11-14 07:17:50]  DEBUG               pyload  ADDON UserAgentSwitcher: Setting maximum redirections to 10
[2023-11-14 07:17:50]  DEBUG               pyload  ADDON UserAgentSwitcher: Use custom user-agent string `Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:87.0) Gecko/20100101 Firefox/87.0`
[2023-11-14 07:17:50]  DEBUG               pyload  DOWNLOADER FilefactoryCom[4533]: Plugin version: 0.64
[2023-11-14 07:17:50]  DEBUG               pyload  DOWNLOADER FilefactoryCom[4533]: Plugin status: testing
[2023-11-14 07:17:50]  WARNING             pyload  DOWNLOADER FilefactoryCom[4533]: Plugin may be unstable
[2023-11-14 07:17:50]  DEBUG               pyload  ADDON ExternalScripts: No script found under folder `download_processed`
[2023-11-14 07:17:50]  INFO                pyload  Download skipped: DLvK-1080-01x03.part3.rar due to File exists

This also happened with multiple files but it might as well just have been a fluke of the moment, since everything else went fine. The HTML it downloaded points towards it having attempted to download multiple files at the same time and so the hoster just passed an HTML file:

DLvK-1080-01x03.part3.rar
500.00 MB uploaded March 24, 2021.

Download error (275)
You have recently started a download. Only Premium members can download multiple files simultaneously. Please try again in 21 mins, 26 secs.

Either way, that's not the point of this request :)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@vuolter vuolter

Labels
feature request New enhancement request pyLoad Next
Milestone
No milestone
Development

No branches or pull requests
2 participants
@mihawk90 @vuolter