[Feature]: Profile resources + loading improvements #1598
Assignees
Labels
enhancement
New feature or request
Comments
ikreymer
added a commit
that referenced
this issue
Mar 15, 2024
- default profile browser resources to either 'profile_browser_cpu' / 'profile_browser_memory' or single browser 'crawler_memory_base' / 'crawler_cpu_base', instead of scaled to the number of browser workers - frontend: check that profile html page is loading, retry if getting nginx error - fixes #1598
ikreymer
added a commit
that referenced
this issue
Mar 15, 2024
- default profile browser resources to either 'profile_browser_cpu' / 'profile_browser_memory' or single browser 'crawler_memory_base' / 'crawler_cpu_base', instead of scaled to the number of browser workers - frontend: check that profile html page is loading, retry if getting nginx error - fixes #1598
ikreymer
added a commit
that referenced
this issue
Mar 15, 2024
- Backend: Use separate resource constraints for profiles: default profile browser resources to either 'profile_browser_cpu' / 'profile_browser_memory' or single browser 'crawler_memory_base' / 'crawler_cpu_base', instead of scaled to the number of browser workers - Frontend: check that profile html page is loading, keep retrying if still getting nginx error instead of loading an iframe with the error. - Fixes #1598 (Prepared for 1.9.4 release)
ikreymer
added a commit
that referenced
this issue
Mar 16, 2024
- Backend: Use separate resource constraints for profiles: default profile browser resources to either 'profile_browser_cpu' / 'profile_browser_memory' or single browser 'crawler_memory_base' / 'crawler_cpu_base', instead of scaled to the number of browser workers - Frontend: check that profile html page is loading, keep retrying if still getting nginx error instead of loading an iframe with the error. Fixes #1598 (Copy of #1599 from 1.9.4)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Context
Currently, profile browser pods have same resource requirements as crawlers. However, profiles only use one tab, while crawlers may have multiple worker tabs. This is slightly wasteful, instead profiles should use equivalent resources to one browser, and also allow custom profile resource overrides, if needed.
Additionally, loading a profile sometimes shows the nginx error page, due to some race condition. To avoid the, frontend can instead retry until the profile page from the container is reached.
What change would you like to see?
Requirements
No response
Todo
No response
The text was updated successfully, but these errors were encountered: