Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Scraping periodically returns potentially cached results. #46

Open
FeraruSilviuMarian opened this issue Dec 23, 2023 · 0 comments
Open

Comments

@FeraruSilviuMarian
Copy link

FeraruSilviuMarian commented Dec 23, 2023

Issue: When posting a new video, scraping returns the correct video id's including the new video, the next scrape returns the previous videos without the new video, the third time scraping returns the correct video ids.

How to reproduce:
Every 70 seconds
videos = scrapetube.get_channel(channel_username = your youtube username)
video_ids = [video['videoId'] for video in videos]
print(video_ids)

Run the script and wait for a print to compare with, then post a video on your youtube channel, and wait for 3+ prints, then compare the results.

I'll use numbers instead of youtube video ID's to demonstrate what results i get, think of the number as a youtube video ID.

[5,4,3,2,1] (state of the channel before new video posted)
[6,5,4,3,2] (new video posted)
[5,4,3,2,1] (scrape now returns the old state of the channel, the previous 5 videos, is this from cache?)
[6,5,4,3,2] (from now on, it returns the correct video ID's)
[6,5,4,3,2]
[6,5,4,3,2]

EDIT: I increased the scrape period to 120 seconds and that worked

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant