-
-
Notifications
You must be signed in to change notification settings - Fork 769
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Feature] Some sites block scraping content without javascript. #6447
Comments
You can try with https://github.com/lwthiker/curl-impersonate/ , which sometimes help. |
Thanks. But how can I combine this with freshrss? |
A typical way is to use a system such as RSS Bridge, which outputs an RSS feed, which can be consumed by FreshRSS. |
Try feedless tool. It can help in some cases. |
Thanks for the above replies. My solution is to use a local headless browser to handle this by python. It is quite light. |
Some sites can not be scraped without javascript. And I tried different useragents such as curl/8.21. All the useragents failed.
Site: https://rsshub.app/zhubai/posts/havefun
The text was updated successfully, but these errors were encountered: