You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The courses now recommend using variety of tools to make HTTP requests. Sometimes it's more confusing, sometimes less.
The got library seems to be superseeded by ky, at least in the README Sindre mentions it.
Apify develops got-scraping, which seems to be married to the got library. Wouldn't it make sense to turn got-scraping into something more agnostic to a client library? Could it just prepare the request details, so that they can be attached to the request by any library?
Using got-scraping in the basic tutorial is probably unnecessary, any HTTP client can be used in the initial lessons. The value of got-scraping should emerge with more complicated use cases.
But wouldn't it make more sense to skip got-scraping and promote Crawlee right away at that point? Is got-scraping something Apify wants to spend marketing energy on, or is it an implementation detail?
Regarding request libraries, the scene is similarly shattered in Python, featuring requests, aiohttp, or httpx, each having their fans and use cases.
I'd like to kick off this as a discussion on what should be the preferred way for the Academy to teach making requests in 2024, using Node.js and Python.
The text was updated successfully, but these errors were encountered:
The courses now recommend using variety of tools to make HTTP requests. Sometimes it's more confusing, sometimes less.
got
library. Wouldn't it make sense to turngot-scraping
into something more agnostic to a client library? Could it just prepare the request details, so that they can be attached to the request by any library?Using
got-scraping
in the basic tutorial is probably unnecessary, any HTTP client can be used in the initial lessons. The value ofgot-scraping
should emerge with more complicated use cases.But wouldn't it make more sense to skip
got-scraping
and promote Crawlee right away at that point? Isgot-scraping
something Apify wants to spend marketing energy on, or is it an implementation detail?As of now,
got-scraping
doesn't have good Python alternatives I'd know about. There are independent libraries one can use, such as fake-user-agent, which have integrations with scraping frameworks.Regarding request libraries, the scene is similarly shattered in Python, featuring requests, aiohttp, or httpx, each having their fans and use cases.
I'd like to kick off this as a discussion on what should be the preferred way for the Academy to teach making requests in 2024, using Node.js and Python.
The text was updated successfully, but these errors were encountered: