Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue with globalFunctions.multithread_download #278

Open
crossflame22 opened this issue Nov 30, 2020 · 2 comments
Open

Issue with globalFunctions.multithread_download #278

crossflame22 opened this issue Nov 30, 2020 · 2 comments

Comments

@crossflame22
Copy link

I've added a few print statements here and there, so I can get a feel for what the code is doing. Here's the code:
`def multithread_download(self, chapter_number, comic_name, comic_url, directory_path, file_names, links, log_flag,
pool_size=4): # ERROR SOMEWHERE IN HERE
"""
:param chapter_number: string used for the progress bar
:param comic_name: string used for the progress bar
:param comic_url: used for the referer
:param directory_path: used to download
:param file_names: files names to download
:param links: links to download
:param log_flag: log flag
:param pool_size: thread pool size, default = 4
:return 0 if no error
"""

    def worker():
        while True:
            print ("Attempting to download") # Returns "0image(s) [00:00, ?image(s)/s]" before it even attempts to download
            try:
                print ("Trying...") # Printed immediately after "Done?"
                worker_item = in_queue.get()
                print ("Sending download request") # Never printed
                self.downloader(referer=comic_url, directory_path=directory_path, pbar=pbar, log_flag=log_flag,
                                image_and_name=worker_item)
                print ("Sent download request") # Never printed
                in_queue.task_done()
            except queue.Empty as ex1:
                print ("queue.Empty as ex1") # Never printed
                logging.info(ex1)
                return
            except Exception as ex:
                print ("Exception as ex") # Never printed
                err_queue.put(ex)
                in_queue.task_done()
            print ("Tried") # Never printed
        print ("Exited loop") # Never printed

    in_queue = queue.Queue()
    err_queue = queue.Queue()

    pbar = tqdm(links, leave=True, unit='image(s)', position=0)
    # pbar.set_description('[Comic-dl] Downloading : %s [%s] ' % (comic_name, chapter_number))

    for i in range(pool_size):
        t = threading.Thread(target=worker)
        t.daemon = True
        t.start()

    for item in zip(links, file_names):
        in_queue.put(item)

    in_queue.join()  # block until all tasks are done

    try:
        err = err_queue.get(block=False)
        pbar.set_description('[Comic-dl] Error : %s [%s] - %s ' % (comic_name, chapter_number, err))
        raise err
    except queue.Empty:
        # pbar.set_description('[Comic-dl] Done : %s [%s] ' % (comic_name, chapter_number))
        print ("Done?") # Printed before it even tries to download
        return 0
    finally:
        pbar.close()`

This returns:
`Starting the script in Verbose Mode

we got here!11!
Got user input https://readcomiconline.to/Comic/The-Amazing-Spider-Man-2015/Issue-1?id=6348
After (potentially) resolving an ID or name, our user input is now https://readcomiconline.to/Comic/The-Amazing-Spider-Man-2015/Issue-1?id=6348
running honcho (??) checker
The comic is on RCO.T!
Starting RCO Download!
Name Cleaner?
Single Chapter detected
Fooling CloudFlare...Please Wait...
Created file directory
DOWNLOADING!!
0image(s) [00:00, ?image(s)/s]Attempting to download
Attempting to downloadTrying...

Trying...Attempting to downloadAttempting to download

Done?Trying...
0image(s) [00:00, ?image(s)/s]

CONVERTING!!
Trying...CBZ File : C:\Users\Owena\Downloads\comic-dl-master\comic-dl-master\comic_dl\The Amazing Spider Man 2015\The Amazing Spider Man 2015 - Ch Issue - 1

[Comic-dl] CBZ File Exist! Skipping : C:\Users\Owena\Downloads\comic-dl-master\comic-dl-master\comic_dl\The Amazing Spider Man 2015\The Amazing Spider Man 2015 - Ch Issue - 1

checker complete
Total Time Taken To Complete : 1.4999992847442627`
I have no idea why, but it returns "Done?" before it ever tries to download, and even then, it never even sends the request to download.

@Xonshiz
Copy link
Owner

Xonshiz commented Sep 4, 2021

Could you please tell me what's the issue first? I see that you were trying to debug... but the issue is unclear to me (sorry).
Would be great if you could download the latest binaries or code and run the script in verbose mode and shared the error log that it generates.

@crossflame22
Copy link
Author

Could you please tell me what's the issue first? I see that you were trying to debug... but the issue is unclear to me (sorry).
Would be great if you could download the latest binaries or code and run the script in verbose mode and shared the error log that it generates.

I'm an idiot and haven't seen this since then. Check my reply to #299 for more info because I'm still having this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants