Year information after title or names and prefetch links when downloading in batch

[devvratmiglani]

It's very hard for Non-Japanese like me to remember names like "Nanatsu no Taizai: Ensa no Edinburgh" and remember which part or chapter it is.
Please add year after title names, it will be great help to Identify what chapter to watch! Thanks in advance!

I mean something like this:

------------------------|------------------------|------------------------|------------------------|------------------------|--------------

Also another thing I noticed that fetching links takes longer when in downloading batch, so can we prefetch links of next episode when an episode is downloading and simultaneously for all, kind of like an optimization.

[port19x]

I guess it would be much easier to remember what comes after the :, not sure if there is much we can do.
We can however definitely prefetch the next link during batch downloads

[devvratmiglani]

I guess it would be much easier to remember what comes after the :, not sure if there is much we can do. We can however definitely prefetch the next link during batch downloads

Yes, you are right I just felt convenience using my last downloader when year is mentioned, but that doesn't work anymore.
Also that downloader saves all links to text file and then performs batch download using aria2c ; It was lightning fast.
I guess prefetch would do it, I mean fetching while downloading.

Link to my last downloader: https://github.com/niveK77pur/gogoanime
(It only fetches episode names; do not fetch links or say it is broken now)

[port19x]

Thanks for referencing that downloader. That way we can inspect it's code

[devvratmiglani]

All the way fine, tbh its author recommended me ani-cli 😄

[CoolnsX]

Thanks for referencing that downloader. That way we can inspect it's code

It uses selenium (out of scope of our project, but a good starting point for anyone coming into web-scraping)

[devvratmiglani]

Well then how do Ani-cli does that without selenium or any automation, cookie handler stuff. Any resources would be a great help.

I only know the way to do is reusing cookies.

[CoolnsX]

simple, using scraping disguising techniques to make the request looks like it's coming from browser like
-- User-agent
-- setting Cipher-suite (for bypassing most of the cloudflare bot protection, except the latest one,it runs JS to calculate something on browser side)
-- Referrer
-- TLS (some cloudflare enabled website also check this)

and many other techniques, like using cookies as you mentioned

[devvratmiglani]

I guess its time to sit on chair for long, Thanks for that.
If any learning resource ,websites where I can get more information, that would be a great help.
Sensei!

[justchokingaround]

the main source i recommending starting with is scraping-tutorial. i take no credit for this repo, i just made a fork of it with a few shellscript examples that i added:
https://github.com/justchokingaround/scraping-tutorial/tree/shellscript

the best resources for webscraping with shellscript imo is already existing code, from which you can get inspired, here are a few useful repos i recommend u take a look at. if u got any questions or need some help just ping, or write to us in the discord server (i'm always glad to help):

https://github.com/CoolnsX/repos_scripts/blob/master/ani-new
https://github.com/sh-dlp/sh-dlp/tree/main/extractors
https://github.com/justchokingaround/zoro-cli/blob/main/zoro.sh

[ninko76]

It's very hard for Non-Japanese like me to remember names like "Nanatsu no Taizai: Ensa no Edinburgh" and remember which part or chapter it is. Please add year after title names, it will be great help to Identify what chapter to watch! Thanks in advance!

I mean something like this:

------------------------|------------------------|------------------------|------------------------|------------------------|--------------

Also another thing I noticed that fetching links takes longer when in downloading batch, so can we prefetch links of next episode when an episode is downloading and simultaneously for all, kind of like an optimization.

What command do you use to download a batch for example?

[devvratmiglani]

It's very hard for Non-Japanese like me to remember names like "Nanatsu no Taizai: Ensa no Edinburgh" and remember which part or chapter it is. Please add year after title names, it will be great help to Identify what chapter to watch! Thanks in advance!
I mean something like this:
------------------------|------------------------|------------------------|------------------------|------------------------|--------------
Also another thing I noticed that fetching links takes longer when in downloading batch, so can we prefetch links of next episode when an episode is downloading and simultaneously for all, kind of like an optimization.

What command do you use to download a batch for example?

this screenshot was with different scr-aper not ani-cli
---> https://github.com/niveK77pur/gogoanime