Some endpoint responses are too large

[firmart]

I ran into a problem where the data returned by the Jikan API was too large, which unfortunately made some endpoints unusable on the project I'm working on, involving LLM. I have getAnimeReviews, getAnimeCharacters, getAnimeRecommendations in mind, but there are others and their "Manga" counterparts.

In my particular use case, a simple additional query parameter limit as already present in the /anime or /manga endpoints will solve the issue, but it would be even nicer to have full pagination. Actually, getAnimeReviews does implement pagination, but the page size cannot be controlled.

Thanks for the great work !

[pushrbx]

Hi @firmart , thanks for the feedback.

The reason there is no pagination for these endpoints is a technical limitation.
All of these endpoints are just scraping the relevant pages from MAL, then they are caching the scrape result in the database for a certain period of time.
If there is no pagination on these MAL pages, we can't add any pagination to these endpoints. (One request to Jikan API is exactly one request to MAL if the page is not cached yet)

A few examples to just make it easy to visualise it:

Reviews
https://myanimelist.net/anime/52991/Sousou_no_Frieren/reviews

There is only a "next page" and "previous page" link button at the bottom of the page.

Characters
https://myanimelist.net/anime/52991/Sousou_no_Frieren/characters
There is no pagination in most of the cases.

Recommendations
https://myanimelist.net/anime/9253/Steins_Gate/userrecs

There is no pagination at all, it just dumps all of them on one page under a collapsible component.

Just to be more helpful:

which unfortunately made some endpoints unusable on the project I'm working on, involving LLM

This statement raises some eyebrows for me. In case of an LLM related project the size of the data should not be a problem, it might be a problem the way you try to parameterise the LLM, or you might want to use a different model which doesn't limit the token count that much. I can give you better hints if you could elaborate on that part of your problem.