-
Notifications
You must be signed in to change notification settings - Fork 65
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(match): Ensemble matching for episodes #668
Open
ShanaryS
wants to merge
4
commits into
master
Choose a base branch
from
ensemble-matching
base: master
Could not load branches
Branch not found: {{ refName }}
Could not load tags
Nothing to show
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+1,164
−396
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ShanaryS
added
feature
New feature or request
search
Related to search
config
related to config file
enhancement
enhances existing features
labels
Apr 27, 2024
zakkarry
reviewed
Apr 28, 2024
ShanaryS
force-pushed
the
ensemble-matching
branch
from
April 28, 2024 01:29
cb9edb0
to
706d777
Compare
zakkarry
reviewed
Apr 28, 2024
ShanaryS
force-pushed
the
ensemble-matching
branch
from
April 28, 2024 14:25
ca252d2
to
b87da75
Compare
ShanaryS
force-pushed
the
ensemble-matching
branch
from
April 28, 2024 21:58
b87da75
to
4e86f6c
Compare
ShanaryS
force-pushed
the
ensemble-matching
branch
from
April 28, 2024 22:21
697e212
to
14ec9e0
Compare
ShanaryS
force-pushed
the
ensemble-matching
branch
from
April 28, 2024 22:27
14ec9e0
to
658f538
Compare
ShanaryS
force-pushed
the
ensemble-matching
branch
from
May 26, 2024 22:08
407b835
to
8e3f27c
Compare
ShanaryS
force-pushed
the
ensemble-matching
branch
from
May 27, 2024 00:42
8e3f27c
to
51bf645
Compare
ShanaryS
force-pushed
the
ensemble-matching
branch
from
May 27, 2024 00:56
51bf645
to
b8cd448
Compare
ShanaryS
force-pushed
the
ensemble-matching
branch
from
May 27, 2024 02:41
b8cd448
to
de216d4
Compare
ShanaryS
force-pushed
the
ensemble-matching
branch
from
May 28, 2024 20:02
de216d4
to
398522b
Compare
ShanaryS
force-pushed
the
ensemble-matching
branch
from
May 28, 2024 21:32
398522b
to
429c5f7
Compare
ShanaryS
force-pushed
the
ensemble-matching
branch
from
May 28, 2024 23:08
429c5f7
to
e6a5ae6
Compare
ShanaryS
force-pushed
the
ensemble-matching
branch
from
May 28, 2024 23:14
e6a5ae6
to
d840295
Compare
ShanaryS
force-pushed
the
ensemble-matching
branch
from
May 28, 2024 23:25
d840295
to
569e7f4
Compare
ShanaryS
force-pushed
the
risky-multi-file-support
branch
from
May 29, 2024 14:58
b4b7922
to
8865364
Compare
ShanaryS
force-pushed
the
risky-multi-file-support
branch
from
May 29, 2024 15:01
8865364
to
589806d
Compare
ShanaryS
force-pushed
the
ensemble-matching
branch
from
May 29, 2024 15:18
569e7f4
to
0849047
Compare
ShanaryS
force-pushed
the
risky-multi-file-support
branch
3 times, most recently
from
May 29, 2024 17:28
b046a69
to
74a217d
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This feature allows grabbing rss/announce and searching of seasons from individual episodes. Along with the changes in v6, this feature completes all possible matches from cross-seed aside from tweaks.
This works by introducing a new type of searchee, a virtual searchee. This is needed since the episodes are sourced from multiple torrents and folders. As such, it hash no
infoHash
orpath
. All areas of the application where these were used for logic has been updated to reflect this new possiblity. In general, this should be treated as a data based searchee if searchee.path is not need. So most checks went fromif (searchee.path)
toif (!searchee.infoHash)
so that virtual searchees are included.Virtual searchees also have absolute file paths instead of relative to make linking easier. This required some small changes when linking as well as changes in perfect matches. For a perfect match, when the item is a virtual searchee it only requires a name match on each file rather than path.
To support rss/announce, the information needed to be cached. I opted to use an in memory database as I did not want to add to the existing which would need migrations. For 2k entries, it takes about ~3s on startup and about 1MB of memory.
This feature works best with search. Due to the nature of the problem, it's easier to find matches by searching rather than from rss/announce, especially for anime.
Also, I added a check to see if the size changed for data based searches and report
TORRENT_NOT_COMPLETE
if it hash. A similar check was done for ensemble but on each file.The implementation is to parse titles and put them in a standard format
cleanTitle.season.resolution.source-RlsGrp
called key. As such two new regexs were added for resolution and source. These are necessary to restrict searching for mixed resolutions and sources. Once there is more than 2 distinct episodes that all have the same key, it's eligible to be searched. The searchee name will be the key while the files will be the absolute path to the largest file in each episode torrent.Some additional checks are put into place. The highest episode number is used as the total number in the season. We then check for the presence of all episodes down to E01 and if more than the
seasonFromEpisodes
is missing, it won't be created. It also checks to ensure that the file with the newest modified time is at least 8 days old since most episodes are released weekly. Of course this will search if there is a mid season break and get no results but the alternative would be to use a sonarr (likely will later). To account for this, if the age of the newest episode is after its last search time, we will search it again immediately (assuming it's more than 8 days old).Note