request() & pagelessPipe()()

[mrWh1te]

Make an API request and return the response

Could potentially make a bot that never interacts with an actual chrome page instance... ie using https://www.reddit.com/dev/api to create a reddit bot

but then it's less about navigating pages, and more about navigating state effected by API calls, functions, etc. Grab data, parse it, make a decision, do something ie make a PUT/POST call to create data in a web app using a publicly shared API.

There is overlap between this and Piping.... not necessarily new stuff entirely. The downside is that you have to create an instance of page to inject it so maybe create a pageless pipe? ie:

await pagelessPipe(
   API('get', 'url'), // get forum threads
   map(transform response to iterative data collection),
   forAll()(
      // for each forum thread, read and respond
      API('post', 'url', 'some newly generated comment')
   )
)(undefined, inject1, inject2, ...) 
// can submit real `page` or nothing, doesn't matter, just to follow `BotAction` interface

This function would be an Assembly-Line BotAction that ignores page provided and injects undefined for page param in assembled BotAction's.

Have to be careful with this and not use it with BotAction's that use page

[mrWh1te]

lowercase instead of uppercase?

await pipe()(
   api('get', 'url'),
   map(transform to iterative collection),
   // .... 
)(page)

[mrWh1te]

Can the pagelessPipe()() BotAction define the page param type as any ?

So you can run it by itself without an actual Puppeteer page instance

[mrWh1te]

What about running API calls from the Puppeteer page via page.evaluate() ? Is that necessary? Is it bad to restrict dev's to running all custom API calls in nodeJS? Maybe in NodeJS by default with the ability to set it in page? What is the benefit(s) from doing it in page?

For some web apps, it could be easier to crawl it by running API calls in page, but without request interception to hydrate request headers, it will be difficult, you'll have to scrape data related to the headers of the calls, and attach manually... but could still be easier than going through a complex UI to write an enriched document, when it could be programtically written and submitted in one API call.

Note, you can use browser extensions with Puppeteer. Still don't see a concrete case, just abstract reasons

[mrWh1te]

in consideration of pagelessPipe()()

what if chain/pipe, by default, did not have page as first param....

instead there would be a page(page)(...actions) injector for injecting page as first param into an assembly line of functionality

then this framework can grow beyond composing web bots -- they can still be "bots" but they are no longer focused solely on automating browser user usage, you could compose async functionality with synchronous functions

edit:
that way the "theme" is maintained, but does that have a negative effect on dev's using it? restrict creativity?

edit2:
but it could be server bots, site api using bots, other protocol bots, whatever bots and unit of functionality could be encapsulated as this automation with the metaphor of being a robot

edit3:
it would be a big refactor, cover more than AbortLineSignal work but not as complicated. Maybe a v3?

edit4:
what are the potential drawbacks of this idea? How much code might it add to the dev using it with Puppeteer?

edit5:
page is then treated like BotFileOptions in files()() as an inject to be injected.. there could be another function for creating a page instance, maybe even using the browser() #45 ... but then how does browser() work (without page 1st)?

edit6:
could potentially generic the first param of BotAction with fallback any followed by spread array so it could focus/center on 1 thing each.... could even generate this code, at least the base level with(out) page for general solution (no specific initial param, just spread injects) and 1 target solution (specific initial param, could even be an injectMap #58, followed by spread injects)

[mrWh1te]

the above would introduce a new BotAction interface with the original BotAction interface that sets page as the first param, maybe called PageBotAction ?

Then the original BotAction interface and many others could drop page from the first param, unless they are expecting it

[mrWh1te]

Could do pagelessPipe()() for v3, in a potentially type hacky way, then leaver the refactor with new BotAction injector page()() for v4

[mrWh1te]

it could be a puppeteer()() BotAction that instantiates the browser, if one not provided, and creates a page, if one not provided for injecting into all assembled BotAction's

Or maybe pull in the #45 browser() BotAction into this thinking

[mrWh1te]

Consider having the pipe object set as the first inject, instead of the last, for more precise typing

[mrWh1te]

In line with #87 architecture discussion for v4, the spread injects array will probably be replaced with a map of some sort that uses 1:1 keys for lookup.

The spread array of injects, with the possibility of a pipe value, while supporting both cases (chain vs pipe); it prevents safe typing throughout.

Also, there is an implicit dependency on BotAction's that rely on injects: the order for which the injects are declared in the function's parameters. This creates complications for dev's who want to mix and match BotAction's with various injects in a single assembly line of BotAction's, ultimately leading to a bunch of unnecessary wrapper functions that remap injects' order.

That said, a final decision has yet to be made.

edit:
To tie it into this topic, a Puppeteer page may be better suited as a key->value pair in the injects map instead of built into the BotAction parametera since it's conceivable and practical to build BotAction's without using it i.e. api().

[mrWh1te]

api()() is going to be #109 for bot communication

therefore, this has to be renamed, perhaps apiCall(), request(), httpRequest() ... ?