YGGTorrent/YGGCookie search doesn't work as long as there is a category selected

[Seluj78]

Is there an existing issue for this?

• I have searched the existing open and closed issues

Current Behavior

If any category is selected (for movies, haven't tried TV shows yet), no results will be returned. But with no categories, results are returned. When searching for a movie from radarr, the category movies is used, which used to work but doesn't anymore.

Expected Behavior

That the releases are reported from YGG with or without the category.

Steps To Reproduce

Using Prowlarr 1.4.1.3243-ls575 by [linuxserver.io](https://www.linuxserver.io/)

• Search for a movie like above with the category Movie
• No results
• Try without
• Results.

Environment

- OS: Linux
- Prowlarr: 1.4.1.3243-ls575 by linuxserver.io 
- Docker Install: Yes
- Using Reverse Proxy: Yes, Traefik (and flaresolverr for YGG)
- Browser: Not related

What branch are you running?

Nightly

Trace Logs?

prowlarr.trace.log

Trace Logs have been provided as applicable. Reports may be closed if the required logs are not provided.

• I have followed the steps in the wiki link above and provided the required trace logs that are relevant and show this issue.

[Seluj78]

The problem with this is that searching automatically AND manually from radarr doesn't work anymore, which is quite bothersome, especially since YGG finally started working again recently

[Seluj78]

Seems to be working fine on sonarr.

[bakerboy448]

Doesn't seem to be a Prowlarr issue?

2023-04-25 23:44:41.9|Info|ReleaseSearchService|Searching indexer(s): [YGGcookie] for Term: [Spider-Man 2 2004], Offset: 0, Limit: 100, Categories: [2000]
2023-04-25 23:44:41.9|Trace|Cardigann|Getting Basic search
2023-04-25 23:44:41.9|Trace|ConfigService|Using default config value for 'logindexerresponse' defaultValue:'False'
2023-04-25 23:44:41.9|Debug|Cardigann|Populating config vars
2023-04-25 23:44:41.9|Debug|Cardigann|Adding request: https://www3.yggtorrent.do/engine/search?do=search&order=desc&sort=publish_date&name=%22Spider%22+%22Man%22+%222%22+%222004%22&category=2145&sub_category=2178
2023-04-25 23:44:41.9|Debug|Cardigann|Downloading Feed https://www3.yggtorrent.do/engine/search?do=search&order=desc&sort=publish_date&name=%22Spider%22+%22Man%22+%222%22+%222004%22&category=2145&sub_category=2178
2023-04-25 23:44:41.9|Trace|ConfigService|Using default config value for 'logindexerresponse' defaultValue:'False'
2023-04-25 23:44:41.9|Trace|IndexerHttpClient|Req: [GET] https://www3.yggtorrent.do/engine/search?do=search&order=desc&sort=publish_date&name=%22Spider%22+%22Man%22+%222%22+%222004%22&category=2145&sub_category=2178
2023-04-25 23:44:41.9|Trace|ConfigService|Using default config value for 'proxyenabled' defaultValue:'False'
2023-04-25 23:44:42.0|Trace|IndexerHttpClient|Res: [GET] https://www3.yggtorrent.do/engine/search?do=search&order=desc&sort=publish_date&name=%22Spider%22+%22Man%22+%222%22+%222004%22&category=2145&sub_category=2178: 200.OK (78673 bytes) (96 ms)
2023-04-25 23:44:42.0|Debug|FlareSolverr|CF Protection not detected, returning original response
2023-04-25 23:44:42.1|Debug|Cardigann|Parsing
2023-04-25 23:44:42.1|Trace|ConfigService|Using default config value for 'logindexerresponse' defaultValue:'False'
2023-04-25 23:44:42.2|Debug|Cardigann|Got 0 releases

There are no releases for that search string in category 2145 sub categories 2178 in the Category you selected to search which appears to have been Movie/Other

https://github.com/Prowlarr/Indexers/blob/master/definitions/v9/yggtorrent.yml#L249-L256

also

[Seluj78]

The thing is, I'm using automatic sync between radarr and prowlarr, and it always worked before...

[Seluj78]

It seems that the automatic sync between radarr and prowlarr forces Radarr to only search with the category Movies/Other. I've successfully forced the indexer in Radarr to not JUST select Movies/Other manually, but as soon as I've set prowlarr to sync again, it stopped working. I'll try and show you what I mean with screenshots

[Seluj78]

Query info from the query made from radarr's interactive search.

Here are the settings for the radarr indexer, they've been automatically set to ONLY movies and movies/other.

If I edit the categories here to only movies or movies and all sub categories, then it will be changed automatically back by prowlarr.

I've disabled the sync between radarr and prowlarr now:

First thing that troubles me is that in the indexer on radarr, with the sync disabled, the only reported categories for movies are movies and movies/other. All the other ones aren't here, even though I've previously selected them all in Prowlarr (before turning off the sync).

If now I do my search again, Only movies is selected as a category but still no result returned

If I click the little search button in the history, same deal, no results.

BUT, if I remove the category it will find results

And then if I re-add the category movies it won't work again

[bakerboy448]

Prowlarr syncs and selects the exact categories you tell it to for the app (advanced option)

YGG is also anti automation and continually taking steps to break prowlarr and jackett, ban users for merely mentioning or discussing them, and continually making life as hard as possible for context.

@mynameisbogdan this fall out from the category and path rework in the definition ?

[Seluj78]

Prowlarr syncs and selects the exact categories you tell it to for the app (advanced option)

Which advanced options are you refering to ? I cannot find these

YGG is also anti automation and continually taking steps to break prowlarr and jackett, ban users for merely mentioning or discussing them, and continually making life as hard as possible for context.

I didn't know that... I did mention prowlarr multiple times on the YGG forums and didn't get banned. But I'll definetly keep that in mind. It's dumb that they're anti automation, it only de-serves them... Automation, at least for my part, only means higher seeding...

So, do you think it is after all related to prowlarr and the definition of the YGG indexer ?

[ilike2burnthing]

When Prowlarr performs a search using the Movie category, it should be making 2 requests - &sub_category=2183 and &sub_category=2178. A search for spider-man 2 or spider-man 2 multi makes both of those requests, but a search for spider-man 2 2004 only requests &sub_category=2178.

When Prowlarr performs a keyword search with no categories, it should be making 2 requests - &category=all twice, the latter time with keywords modified (S01 > Saison 1). A search for spider-man S02 does this, but a search for spider-man 2 also requests &category=2145.

When Prowlarr performs a keywordless search with no categories, it should be making 2 requests - &category=all and &page=50&category=all. However, it seems to be making every possible category search, returning ~600 results.

At a guess, there's an issue with Prowlarr's use of search paths categories, though I'm not sure why it's dependent on particular keywords.

[ilike2burnthing]

Bit of an aside. I noticed that when manually searching with 1 indexer selected all possible categories are still listed, rather than only those supported by that indexer, and indexer-specific categories (>=100000) are also missing.

Implementing the latter was rejected in #722, but I couldn't see any issues for the former. Would that be something you'd be open to?

[Seluj78]

Just a heads up, for keywords, I was using spider man 2 as an example, but any search (for a movie that exists within YGG) won't work the same way. I've discovered this while searching for les volets verts.

I just chose Spider man 2 as it was a universal thing.

[mynameisbogdan]

@ilike2burnthing this is by design in prowlarr. The paths are "pageable", meaning it stops when 0 releases are found. Thus skipping the others. The logic is simple: if aren't any results on page 1, don't fetch page 2, etc.

Searching for spider man 2 works and spider man 2 2004 not, it's simply because the first page (name="spider"+"man"+"2"&category=2145&sub_category=2178) has one non-relevant release from 2018.

[Seluj78]

So this is a bug then ?

[ilike2burnthing]

That makes sense if it's just pages, but cases like this are then just broken by it.

[Seluj78]

Any news on this ? 😊

[Lacoste313]

Same problem here.
As i can see its a sub_category problem on the request URL from prowlarr.

For exemple the request url (find in prowlarr.log):
https://www3.yggtorrent.do/engine/search?do=search&order=desc&sort=publish_date&name=%22Die%22+%22Hard%22+%222%22+%221990%22&category=2145&sub_category=2178
Dont find anything with sub_category set.

But, without setting the sub_category. A result is found:
https://www3.yggtorrent.do/engine/search?do=search&order=desc&sort=publish_date&name=%22Die%22+%22Hard%22+%222%22+%221990%22&category=2145

A tips exist for removing the sub_category automatically in the request URL ?

I go to make some test.

[Lacoste313]

All its ok for me.

FIX:

You need to modify the "Sync Categories" in prowlarr (Setting >> app >> radarr)
To just have the category "movie" selected and any sub.(eg: :Movies/HD)

If you can uncheck sub category, you need to uncheck first the category (Movies) and uncheck all sub after. (dont forget to re-check the category Movies after thats)

You maybe need to make the same in radarr, depend your sync setting

and voila

Thats all ✌️

[Seluj78]

All its ok for me.

FIX:

You need to modify the "Sync Categories" in prowlarr (Setting >> app >> radarr)
To just have the category "movie" selected and any sub.(eg: :Movies/HD)

If you can uncheck sub category, you need to uncheck first the category (Movies) and uncheck all sub after. (dont forget to re-check the category Movies after thats)

You maybe need to make the same in radarr, depend your sync setting

and voila

Thats all ✌️

This works for a few until a sync is done between prowlarr and radarr, so I think a fix from prowlarr is needed

[Lacoste313]

This works for a few until a sync is done between prowlarr and radarr, so I think a fix from prowlarr is needed

For the moment all work fine 18h~ after the trick.
Maybe no sync is done in this time i dont know. . I need to wait more time for checking.

[ilike2burnthing]

There is no 'trick', it has nothing to do with sync settings.

[Lacoste313]

There is no 'trick', it has nothing to do with sync settings.

prowlarr is set a full sync with radarr and sonarr. I dont talk about this sync setting.

Look the sync caterogy in my screenshot. I talk about that when a say trick.

[ilike2burnthing]

Again, it has nothing to do with sync settings (including categories).

[Lacoste313]

Hmm okay. Sorry for my mistake.

for the moment all its ok for me without sub category.

How many time you would say when u say « this work for a few until sync »?

[Seluj78]

For me, your change didn't fix it. I cannot get Prowlarr to report anything as long as there is a category selected:

Only movies

Nothing

Movies + sub

Only sub

[Seluj78]

Can someone explain what the problem is? I could do a PR if I understand what's happening and how to fix it.

[ilike2burnthing]

#1628 (comment)
#1628 (comment)
#1628 (comment)

[Seluj78]

#1628 (comment) #1628 (comment) #1628 (comment)

This doesn't help me I'm afraid

[ilike2burnthing]

The logic for Jackett is perform all relevant searches, depending on specified categories.

The logic for Prowlarr (as far as I understand) is perform all relevant searches, depending on specified categories, BUT stop when the first one returns no results. So if the categories selected mean that three different searches should be performed, but the first one returns no results, the other two won't be performed.

That logic makes sense when it's applied to pages (e.g. if page 1 returns no results, don't request page 2), but here we're making completely different searches, so it breaks the indexer (and likely a few others).

[Seluj78]

Do you think you could fix it @ilike2burnthing ?

[ilike2burnthing]

Nope, not a chance.

[Seluj78]

Is there a way to introduce a quick fix to allow indexers to have no category selected ? Since it seems like it works

[Seluj78]

How about @bakerboy448 and @mynameisbogdan ? Cause this is quite the bottleneck

[mynameisbogdan]

There's no quick fix, this has to be done in both jackett and prowlarr in the same time which implies a lot of work since it's an another schema.json change (eg. adding boolean Search.Pageable-ish) and both work VERY differently.

If it was easy fix, trust me it would be already fixed.

Also kindly, there's no need to mention me since I'm already up to date with this and it's under my radar since it affects some of my indexers too.

[Seluj78]

Alright, sorry ! Thank you for the explanation. Has an issue been opened on Jackett too ?

[mynameisbogdan]

No, we're already aware of the issue. hint: me & @ilike2burnthing

[Seluj78]

Ah I see you're contributing to both repositories ! Thanks again and good luck trying to fix this. Do let me know if you need help/testing

[ilike2burnthing]

I can PR a temp fix reverting the relevant parts of Prowlarr/Indexers@0bbd604. YGG would need to be maintained separately from Jackett's versions until a full fix is in place (edit: it kinda already is, because of the FlareSolverr info note).

[ilike2burnthing]

Prowlarr/Indexers#333

[ptichalouf]

ygg doesn't work again sadly

[bakerboy448]

Please use one of the support channels: Discord for support/questions.