Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multiscrape ‘slow’ site #361

Open
Paul-Vdp opened this issue Apr 19, 2024 · 0 comments
Open

Multiscrape ‘slow’ site #361

Paul-Vdp opened this issue Apr 19, 2024 · 0 comments

Comments

@Paul-Vdp
Copy link

I am trying to scrape my provider’s account/subscription status page, but failing big time.
I am pretty sure this is because it must be the slowest site in the universe. Although it only provides me with a rather simple page, it takes very long to show completely - easily up to half a minute. It seems to be generated in at least three stages, automatically refreshing after every stage to show some additional info.
I think the problem is that right after getting to the first stage being shown, Multiscrape goes to work but totally fails because the information I’m after is not yet available.
So, the question is : can Multiscrape be told to WAIT for the full page to be shown, by monitoring the availability of some selector or other token ? Even a simple wait-time parameter would help, because the total scraping speed is of no importance to me (one scrape per day is all I need).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant