Skip to content
This repository has been archived by the owner on Apr 9, 2023. It is now read-only.

Optimization of Puppeteer #456

Open
Givemeurcookies opened this issue Jun 14, 2021 · 0 comments
Open

Optimization of Puppeteer #456

Givemeurcookies opened this issue Jun 14, 2021 · 0 comments

Comments

@Givemeurcookies
Copy link

Hey!
I've previously used Puppeteer for web scraping and I am looking into pre-rendering a few thousand pages in a project. I see this isn't recommended in the README.md due to RAM usage however I don't believe Puppeteer used much RAM even when scraping 50+ pages every second.

I've tried to wrap my head around this project, but puppeteer is imported and controlled from this file in another repo, is this correct?
https://github.com/JoshTheDerf/prerenderer/blob/master/renderers/renderer-puppeteer/es6/renderer.js

I can see that third party pages can be intercepted and blocked but is there a reason why images, media and fonts aren't intercepted and blocked? Those might be the cause of the large RAM usage and shouldn't be needed to render the page properly. Doing so would also potentially speed up the pre-rendering as the data doesn't have to be loaded.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant