Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(config/searchLimit): Perform searches on indexers independently #566

Closed
wants to merge 8 commits into from
Closed

feat(config/searchLimit): Perform searches on indexers independently #566

wants to merge 8 commits into from

Conversation

ShanaryS
Copy link
Contributor

@ShanaryS ShanaryS commented Dec 12, 2023 •
edited by zakkarry

Closes: #565

Very simple change, I simply created a Map<indexerId, count> to keep track. Added to sample configs as well as an info log to at start of search if enabled, and verbose logs for search on each indexer. I've tested it and it works.

I have never written javascript and just wrote from my experience with other languages so please be careful.

2023-12-12 17:21:30 �[32minfo�[39m: [scheduler] starting job: search
2023-12-12 17:23:25 �[32minfo�[39m: [search] Found 4183 torrents, 3981 suitable to search for matches
2023-12-12 17:23:25 �[32minfo�[39m: [search] Limited to 250 searches
2023-12-12 17:23:25 �[32minfo�[39m: [search] Limited to 4 searches per indexer
.
.
.
2023-12-12 17:24:14 �[36mverbose�[39m: [torznab] [4/4] Indexer http://qserver:9696/34/api has reached search limit
2023-12-12 17:24:14 �[36mverbose�[39m: [torznab] [1/4] Indexer http://qserver:9696/27/api searches
2023-12-12 17:24:14 �[32minfo�[39m: [torznab] Searching 1 indexers for 65 (2023) br hdr10 10bit hevc-d3g (filtered by timestamps)
2023-12-12 17:24:14 �[36mverbose�[39m: [torznab] http://qserver:9696/27/api?apikey=[REDACTED]&t=search&q=65+2023

@zakkarry
Copy link
Collaborator

zakkarry commented Dec 13, 2023 •
edited

We're pretty cautious about adding config settings, so it might be better to default to this behavior than make it a setting if we do implement this, I'm not even sure what the benefit of not using this would be in a real use-case.

Do you have an argument for why this should be an option and not the default behavior?

Edit: Oh I see what you're doing, you're using an overall limit for total, and limiting per indexer.
before reviewing the log you posted and stuff, I thought you were just limiting it to the same.
But to my point, I'm not sure why we wouldn't just use searchLimit as it is, and limit each indexer to that many searches, when one hits that limit, it stops searching. In most cases, this would keep the behavior effectively the same except in cases where a indexer is timed out and the timeout is less than the duration of the total search.

I think that would be more the way I would go...

@zakkarry zakkarry added feature New feature or request search Related to search config related to config file labels Dec 13, 2023
@ShanaryS
Copy link
Contributor Author

I agree, there is no practical reason not to make it default. I updated the changes so now searchLimit now applies per indexer rather than overall.

@ShanaryS ShanaryS changed the title Added searchLimitIndexer option to limit searches per indexer Updated searchLimit to limit searches per indexer Dec 13, 2023
@zakkarry zakkarry changed the title Updated searchLimit to limit searches per indexer feat(config/searchLimit): applies limit on per-indexer basis Dec 15, 2023
@ShanaryS
Copy link
Contributor Author

Added a new feature explained here: #565 (comment)

@zakkarry
Copy link
Collaborator

Added a new feature explained here: #565 (comment)

That seems to be out of the scope of your original PR?

@ShanaryS
Copy link
Contributor Author

I can make a new issue and PR if necessary, but I added it on this branch as it needs the searchLimit changes. Also has the same motivation as the original so I thought it would be easier to just do it at all at once.

@ShanaryS ShanaryS changed the title feat(config/searchLimit): applies limit on per-indexer basis feat(config/searchLimit): Perform searches on indexers independently Feb 19, 2024
@mmgoodnow
Copy link
Collaborator

Thanks for the contribution—I appreciate your initiative here and I'm glad you were able to find a solution that works for you.

Ultimately, your use case seems to be pretty uncommon and I don't think it's worth the added complexity and maintenance burden for us to merge this PR as native functionality of cross-seed. If we hear about this issue more in the future, we'll revisit this decision.

@mmgoodnow mmgoodnow closed this Feb 19, 2024
@SnippetSpace SnippetSpace mentioned this pull request Mar 6, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
config related to config file feature New feature or request search Related to search
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Perform searches on indexers independently
3 participants
@ShanaryS @zakkarry @mmgoodnow