Exclude movies in collections created in Plex

[stixx]

Describe the bug
I don't know if this is actually a bug but what I'm trying to accomplish is somewhat the same as the following feature request: #493

I want to create a rule to exclude all movies that are in collections made in Plex. I could be using the Radarr tag to exclude movies but that is rather tedious with large libraries.

Maybe I'm doing something wrong, if so please help me out :).

To Reproduce
Create a rule with the following criteria (see screenshot):

• First value: Present in amount of other collections
• Action: Equals
• Second value: Number
• Custom value: 0

If this rule is not supposed to work this way than this bug report should be a feature request.

Expected behavior
Created collection from rule should not contain movies which are in collections created within Plex.

Screenshots

Device (please complete the following information):

• Docker: latest image

        "Id": "sha256:b02867c7205dcf01990d20fc4c68fc9157d5841f6e98bdb216b0456dd5834465",
        "RepoTags": [
            "jorenn92/maintainerr:latest"
        ],
        "RepoDigests": [
            "jorenn92/maintainerr@sha256:a53573d2366c4908e8cbd5df327a7062a27e767124e68a47714071d6979c94c8"
        ],

[jorenn92]

Hey! Sorry for the late reply.

There's an issue with the 'Plex - Present in amount of other collections' rule in release 1.7.0, which is why it doesn't work as expected. It's currently fixed in the develop branch and will be available in the next release.

Once the rule is functioning, it should achieve what you desire.

[xbglowx]

@jorenn92 will that also fix my problem? #691

[bolmstedt]

This bug seems to still be unresolved in 1.7.1. When I run a rule for movies present in 0 other collections, it adds a movie that matches the rest of the rules to my custom collection, but on the next run it removes that movie from the collection, and then it just keeps cycling like that for consecutive runs, adding it and removing it from the collection.

It does keep a movie that does not match the rules at all which I added manually to the collection in Plex, if that is helpful.

I've had a look at the source code, and it seems to be correct, but it does not seem to work for some reason.

Update:
I just tried to remove the other collections rule from it's separate section and it to the first section, and now it seems to be working. That would mean that there is a bug with how sections interact, or I do not yet understand how sections are designed to work.

[jorenn92]

This bug seems to still be unresolved in 1.7.1. When I run a rule for movies present in 0 other collections, it adds a movie that matches the rest of the rules to my custom collection, but on the next run it removes that movie from the collection, and then it just keeps cycling like that for consecutive runs, adding it and removing it from the collection.

It does keep a movie that does not match the rules at all which I added manually to the collection in Plex, if that is helpful.

I've had a look at the source code, and it seems to be correct, but it does not seem to work for some reason.

Update: I just tried to remove the other collections rule from it's separate section and it to the first section, and now it seems to be working. That would mean that there is a bug with how sections interact, or I do not yet understand how sections are designed to work.

Can you provide a screenshot of your original (non-working) rule? There might be a logical explanation for the observed behavior. If not, I'll attempt to replicate it for further investigation.

[bolmstedt]

Actually, it is still happening, but for a second movie, not the first. I believe it might be related to the cached metadata not being updated correctly, or something.

This is my current config:

[jorenn92]

Actually, it is still happening, but for a second movie, not the first. I believe it might be related to the cached metadata not being updated correctly, or something.

This is my current config:

If you're willing to experiment with a preview build and are comfortable using something like Postman or Insomnia, you might want to explore a feature I've been developing. While the backend code is complete, please be aware that the UI component is not, so for now it's only callable through the API.

This feature allows you to test a particular media item with your rule, providing a detailed breakdown of each evaluation along with the compared values. This could be valuable in understanding the actions applied to your failing media items. If you're interested, give it a try and let me know your observations or any feedback you may have.

To give it a try, follow these instructions:

1. Update to the jorenn92/maintainerr:feature-preview docker image. Be aware of a breaking change related to security. Ensure that your exposed database file is read/writeable by PID 1000:1000. You can specify your own PID in the docker-compose file by adding a user: 1000:1000 record with the desired PID:GID values. Failure to do so may result in Maintainerr being unable to write to the database.

2. Navigate to the collections screen, click on your collection & use the Test media feature.

2. Identify the Plex ID of the media item you want to inspect. The easiest method is to open your browser's developer tools, navigate to the network tab, clear previous records, and use the search functionality on Maintainerr's overview screen to filter for the desired item. Open the server request related to your search, click the 'response' tab, and copy the 'ratingKey' value. Alternatively, you may also use Plex Web and use the developer tools 'network' screen when opening the UI of the desired item.

3. Find the ruleGroup ID by opening the developer tools (clear all previous requests) and open the 'rules' screen. The response of the 'rules' request contains an object for each rule. Copy the 'id' value of the rule you wish to check.

4. Utilize your preferred API testing tool to perform a POST call to 'http://<your_server>:<your_port>/api/rules/test' (for instance, http://192.168.0.1:8154/api/rules/test) with the following JSON request:

5. Copy and analyze the response, which should contain the results of the executed steps. Note that sections won't be evaluated further if it's evident that the item doesn't match the ruleset.

6. When you're done, ensure that you revert to the latest image. This preview image is specifically built for testing features that are not yet ready to be merged into the main branch (and certainly not prepared for release).

[jorenn92]

I've updated the above instructions since there's a basic UI available now. You won't need any external apps anymore.

[bolmstedt]

Sorry for the delay, but here is what I'm seeing.

This is for an item that stays in the collection between runs:

- plexId: 10711
  result: false
  sectionResults:
    - id: 0
      result: false
      ruleResults:
        - operator: OR
          action: not_equals
          firstValueName: Plex - User rating (scale 1-10)
          firstValue: 3
          secondValueName: number
          secondValue: 0
          result: true
        - operator: AND
          action: smaller
          firstValueName: Plex - User rating (scale 1-10)
          firstValue: 3
          secondValueName: number
          secondValue: 6
          result: true
        - operator: AND
          action: equals
          firstValueName: Plex - Present in amount of other collections
          firstValue: 1
          secondValueName: number
          secondValue: 0
          result: false

This is for an item that is added or removed from the collection every run while currently in the collection:

- plexId: 10862
  result: true
  sectionResults:
    - id: 0
      result: true
      ruleResults:
        - operator: OR
          action: not_equals
          firstValueName: Plex - User rating (scale 1-10)
          firstValue: 4
          secondValueName: number
          secondValue: 0
          result: true
        - operator: AND
          action: smaller
          firstValueName: Plex - User rating (scale 1-10)
          firstValue: 4
          secondValueName: number
          secondValue: 6
          result: true
        - operator: AND
          action: equals
          firstValueName: Plex - Present in amount of other collections
          firstValue: 0
          secondValueName: number
          secondValue: 0
          result: true

This is for an item that is added or removed from the collection every run while not currently in the collection:

- plexId: 23259
  result: false
  sectionResults:
    - id: 0
      result: false
      ruleResults:
        - operator: OR
          action: not_equals
          firstValueName: Plex - User rating (scale 1-10)
          firstValue: 4
          secondValueName: number
          secondValue: 0
          result: true
        - operator: AND
          action: smaller
          firstValueName: Plex - User rating (scale 1-10)
          firstValue: 4
          secondValueName: number
          secondValue: 6
          result: true
        - operator: AND
          action: equals
          firstValueName: Plex - Present in amount of other collections
          firstValue: 1
          secondValueName: number
          secondValue: 0
          result: false

This is the logs that produced the previous state:

maintainerr  | [Nest] 76  - 01/23/2024, 4:08:01 PM     LOG [RuleExecutorService] Starting Execution of all active rules.
maintainerr  | [Nest] 76  - 01/23/2024, 4:08:01 PM     LOG [RuleExecutorService] Executing rules for 'Low rated movies'
maintainerr  | [Nest] 76  - 01/23/2024, 4:08:21 PM     LOG [RuleExecutorService] Removing 1 media items from 'Leaving soon'.
maintainerr  | [Nest] 76  - 01/23/2024, 4:08:21 PM     LOG [RuleExecutorService] Adding 1 media items to 'Leaving soon'.
maintainerr  | [Nest] 76  - 01/23/2024, 4:08:21 PM     LOG [CollectionsService] Adding media with id 10862 to collection..
maintainerr  | [Nest] 76  - 01/23/2024, 4:08:22 PM     LOG [CollectionsService] Removing media with id 23259 from collection..

Running the rules again produces these logs:

maintainerr  | [Nest] 76  - 01/23/2024, 4:13:42 PM     LOG [RuleExecutorService] Starting Execution of all active rules.
maintainerr  | [Nest] 76  - 01/23/2024, 4:13:43 PM     LOG [RuleExecutorService] Executing rules for 'Low rated movies'
maintainerr  | [Nest] 76  - 01/23/2024, 4:14:02 PM     LOG [RuleExecutorService] Removing 1 media items from 'Leaving soon'.
maintainerr  | [Nest] 76  - 01/23/2024, 4:14:02 PM     LOG [RuleExecutorService] Adding 1 media items to 'Leaving soon'.
maintainerr  | [Nest] 76  - 01/23/2024, 4:14:02 PM     LOG [CollectionsService] Adding media with id 23259 to collection..
maintainerr  | [Nest] 76  - 01/23/2024, 4:14:02 PM     LOG [CollectionsService] Removing media with id 10862 from collection..
maintainerr  | [Nest] 76  - 01/23/2024, 4:14:02 PM     LOG [RuleExecutorService] Execution of rules for 'Low rated movies' done.
maintainerr  | [Nest] 76  - 01/23/2024, 4:14:02 PM     LOG [RuleExecutorService] Synced collection 'Leaving soon' with Plex

Testing the media again after that run shows the opposite, the result is true for 23259 and false for 10862.

[jorenn92]

Sorry for the delay, but here is what I'm seeing.

This is for an item that stays in the collection between runs:

- plexId: 10711
  result: false
  sectionResults:
    - id: 0
      result: false
      ruleResults:
        - operator: OR
          action: not_equals
          firstValueName: Plex - User rating (scale 1-10)
          firstValue: 3
          secondValueName: number
          secondValue: 0
          result: true
        - operator: AND
          action: smaller
          firstValueName: Plex - User rating (scale 1-10)
          firstValue: 3
          secondValueName: number
          secondValue: 6
          result: true
        - operator: AND
          action: equals
          firstValueName: Plex - Present in amount of other collections
          firstValue: 1
          secondValueName: number
          secondValue: 0
          result: false

This is for an item that is added or removed from the collection every run while currently in the collection:

- plexId: 10862
  result: true
  sectionResults:
    - id: 0
      result: true
      ruleResults:
        - operator: OR
          action: not_equals
          firstValueName: Plex - User rating (scale 1-10)
          firstValue: 4
          secondValueName: number
          secondValue: 0
          result: true
        - operator: AND
          action: smaller
          firstValueName: Plex - User rating (scale 1-10)
          firstValue: 4
          secondValueName: number
          secondValue: 6
          result: true
        - operator: AND
          action: equals
          firstValueName: Plex - Present in amount of other collections
          firstValue: 0
          secondValueName: number
          secondValue: 0
          result: true

This is for an item that is added or removed from the collection every run while not currently in the collection:

- plexId: 23259
  result: false
  sectionResults:
    - id: 0
      result: false
      ruleResults:
        - operator: OR
          action: not_equals
          firstValueName: Plex - User rating (scale 1-10)
          firstValue: 4
          secondValueName: number
          secondValue: 0
          result: true
        - operator: AND
          action: smaller
          firstValueName: Plex - User rating (scale 1-10)
          firstValue: 4
          secondValueName: number
          secondValue: 6
          result: true
        - operator: AND
          action: equals
          firstValueName: Plex - Present in amount of other collections
          firstValue: 1
          secondValueName: number
          secondValue: 0
          result: false

This is the logs that produced the previous state:

maintainerr  | [Nest] 76  - 01/23/2024, 4:08:01 PM     LOG [RuleExecutorService] Starting Execution of all active rules.
maintainerr  | [Nest] 76  - 01/23/2024, 4:08:01 PM     LOG [RuleExecutorService] Executing rules for 'Low rated movies'
maintainerr  | [Nest] 76  - 01/23/2024, 4:08:21 PM     LOG [RuleExecutorService] Removing 1 media items from 'Leaving soon'.
maintainerr  | [Nest] 76  - 01/23/2024, 4:08:21 PM     LOG [RuleExecutorService] Adding 1 media items to 'Leaving soon'.
maintainerr  | [Nest] 76  - 01/23/2024, 4:08:21 PM     LOG [CollectionsService] Adding media with id 10862 to collection..
maintainerr  | [Nest] 76  - 01/23/2024, 4:08:22 PM     LOG [CollectionsService] Removing media with id 23259 from collection..

Running the rules again produces these logs:

maintainerr  | [Nest] 76  - 01/23/2024, 4:13:42 PM     LOG [RuleExecutorService] Starting Execution of all active rules.
maintainerr  | [Nest] 76  - 01/23/2024, 4:13:43 PM     LOG [RuleExecutorService] Executing rules for 'Low rated movies'
maintainerr  | [Nest] 76  - 01/23/2024, 4:14:02 PM     LOG [RuleExecutorService] Removing 1 media items from 'Leaving soon'.
maintainerr  | [Nest] 76  - 01/23/2024, 4:14:02 PM     LOG [RuleExecutorService] Adding 1 media items to 'Leaving soon'.
maintainerr  | [Nest] 76  - 01/23/2024, 4:14:02 PM     LOG [CollectionsService] Adding media with id 23259 to collection..
maintainerr  | [Nest] 76  - 01/23/2024, 4:14:02 PM     LOG [CollectionsService] Removing media with id 10862 from collection..
maintainerr  | [Nest] 76  - 01/23/2024, 4:14:02 PM     LOG [RuleExecutorService] Execution of rules for 'Low rated movies' done.
maintainerr  | [Nest] 76  - 01/23/2024, 4:14:02 PM     LOG [RuleExecutorService] Synced collection 'Leaving soon' with Plex

Testing the media again after that run shows the opposite, the result is true for 23259 and false for 10862.

Thank you for providing this detailed report!

Regarding the item that remains in the collection, could you attempt to manually remove it? Subsequently, remove the exclusion for the item that was created due to this action. It is possible that Maintainerr assumes the record was added manually. If the item no longer appears after these actions, that likely confirms the case.

As for the other items, it seems they are consistently shifting between collections. They seem to be added to one collection, then removed from another, and vice versa—all because the condition 'Plex - Present in amount of other collections' no longer aligns.

To troubleshoot, consider adding the 'Plex - collections media is present in' rule to your rule set. Just for testing purposes, it will include the collection names in the test output. This may shed light on the shifting behavior of these items.

[bolmstedt]

Adding that additional rule gives some interesting insight, I think:

For the item (10711) that always stays in the collection:

...
        - operator: AND
          action: equals
          firstValueName: Plex - Present in amount of other collections
          firstValue: 1
          secondValueName: number
          secondValue: 0
          result: false
        - operator: OR
          action: equals
          firstValueName: Plex - [list] Collections media is present in (titles)
          firstValue:
            - Leaving soon
          secondValueName: text
          secondValue: foobar
          result: false

For the shifting item (10862) that is currently in the collection:

...
        - operator: AND
          action: equals
          firstValueName: Plex - Present in amount of other collections
          firstValue: 0
          secondValueName: number
          secondValue: 0
          result: true
        - operator: OR
          action: equals
          firstValueName: Plex - [list] Collections media is present in (titles)
          firstValue: null
          secondValueName: text
          secondValue: foobar
          result: false

For the shifting item (23259) that is currently not in the collection:

...
        - operator: AND
          action: equals
          firstValueName: Plex - Present in amount of other collections
          firstValue: 1
          secondValueName: number
          secondValue: 0
          result: false
        - operator: OR
          action: equals
          firstValueName: Plex - [list] Collections media is present in (titles)
          firstValue:
            - Leaving soon
          secondValueName: text
          secondValue: foobar
          result: false

You would expect that the first and second item would have the same results for that ruleset, because both are currently in the "Leaving soon" collection, but it's the one currently not in the collection that has the same results.

This brings me back to my initial idea about the cached metadata might be out of sync, which might cause this function to then return inaccurate data:
https://github.com/jorenn92/Maintainerr/blob/main/server/src/modules/rules/getter/plex-getter.service.ts#L97-L111

Now, back to what you first suggested with removing that item (10711) that stayed in the collection.

When it's not in the collection (before rules have run):

...
        - operator: AND
          action: equals
          firstValueName: Plex - Present in amount of other collections
          firstValue: 0
          secondValueName: number
          secondValue: 0
          result: true
        - operator: OR
          action: equals
          firstValueName: Plex - [list] Collections media is present in (titles)
          firstValue: null
          secondValueName: text
          secondValue: foobar
          result: false

Running the rule added it to the collection (as expected), but checking it again looks exactly like before, like the cached metadata was not updated:

...
        - operator: AND
          action: equals
          firstValueName: Plex - Present in amount of other collections
          firstValue: 0
          secondValueName: number
          secondValue: 0
          result: true
        - operator: OR
          action: equals
          firstValueName: Plex - [list] Collections media is present in (titles)
          firstValue: null
          secondValueName: text
          secondValue: foobar
          result: false

Running the rule removes it from the collection (I guess this is what I expected?), and it's back to looking like the other switching item:

...
        - operator: AND
          action: equals
          firstValueName: Plex - Present in amount of other collections
          firstValue: 1
          secondValueName: number
          secondValue: 0
          result: false
        - operator: OR
          action: equals
          firstValueName: Plex - [list] Collections media is present in (titles)
          firstValue:
            - Leaving soon
          secondValueName: text
          secondValue: foobar
          result: false

Re-running the rule now adds it or removes it (and another of the switching items). Here are the logs, just for good measure:

maintainerr  | [Nest] 76  - 01/25/2024, 12:22:41 PM     LOG [RulesService] Removed exclusion for media with id 10711 and rulegroup id 9
maintainerr  | [Nest] 76  - 01/25/2024, 12:24:56 PM     LOG [RuleExecutorService] Starting Execution of all active rules.
maintainerr  | [Nest] 76  - 01/25/2024, 12:24:56 PM     LOG [RuleExecutorService] Executing rules for 'Low rated movies'
maintainerr  | [Nest] 76  - 01/25/2024, 12:25:13 PM     LOG [RuleExecutorService] Removing 1 media items from 'Leaving soon'.
maintainerr  | [Nest] 76  - 01/25/2024, 12:25:13 PM     LOG [RuleExecutorService] Adding 2 media items to 'Leaving soon'.
maintainerr  | [Nest] 76  - 01/25/2024, 12:25:14 PM     LOG [CollectionsService] Adding media with id 10711 to collection..
maintainerr  | [Nest] 76  - 01/25/2024, 12:25:14 PM     LOG [CollectionsService] Adding media with id 23259 to collection..
maintainerr  | [Nest] 76  - 01/25/2024, 12:25:14 PM     LOG [CollectionsService] Removing media with id 10862 from collection..
maintainerr  | [Nest] 76  - 01/25/2024, 12:25:14 PM     LOG [RuleExecutorService] Execution of rules for 'Low rated movies' done.
maintainerr  | [Nest] 76  - 01/25/2024, 12:25:14 PM     LOG [RuleExecutorService] Synced collection 'Leaving soon' with Plex
maintainerr  | [Nest] 76  - 01/25/2024, 12:26:51 PM     LOG [RuleExecutorService] Starting Execution of all active rules.
maintainerr  | [Nest] 76  - 01/25/2024, 12:26:51 PM     LOG [RuleExecutorService] Executing rules for 'Low rated movies'
maintainerr  | [Nest] 76  - 01/25/2024, 12:27:08 PM     LOG [RuleExecutorService] Removing 2 media items from 'Leaving soon'.
maintainerr  | [Nest] 76  - 01/25/2024, 12:27:08 PM     LOG [RuleExecutorService] Adding 1 media items to 'Leaving soon'.
maintainerr  | [Nest] 76  - 01/25/2024, 12:27:08 PM     LOG [CollectionsService] Adding media with id 10862 to collection..
maintainerr  | [Nest] 76  - 01/25/2024, 12:27:09 PM     LOG [CollectionsService] Removing media with id 10711 from collection..
maintainerr  | [Nest] 76  - 01/25/2024, 12:27:09 PM     LOG [CollectionsService] Removing media with id 23259 from collection..
maintainerr  | [Nest] 76  - 01/25/2024, 12:27:09 PM     LOG [RuleExecutorService] Execution of rules for 'Low rated movies' done.
maintainerr  | [Nest] 76  - 01/25/2024, 12:27:09 PM     LOG [RuleExecutorService] Synced collection 'Leaving soon' with Plex
maintainerr  | [Nest] 76  - 01/25/2024, 12:29:25 PM     LOG [RuleExecutorService] Starting Execution of all active rules.
maintainerr  | [Nest] 76  - 01/25/2024, 12:29:25 PM     LOG [RuleExecutorService] Executing rules for 'Low rated movies'
maintainerr  | [Nest] 76  - 01/25/2024, 12:29:42 PM     LOG [RuleExecutorService] Removing 1 media items from 'Leaving soon'.
maintainerr  | [Nest] 76  - 01/25/2024, 12:29:42 PM     LOG [RuleExecutorService] Adding 2 media items to 'Leaving soon'.
maintainerr  | [Nest] 76  - 01/25/2024, 12:29:42 PM     LOG [CollectionsService] Adding media with id 10711 to collection..
maintainerr  | [Nest] 76  - 01/25/2024, 12:29:43 PM     LOG [CollectionsService] Adding media with id 23259 to collection..
maintainerr  | [Nest] 76  - 01/25/2024, 12:29:43 PM     LOG [CollectionsService] Removing media with id 10862 from collection..
maintainerr  | [Nest] 76  - 01/25/2024, 12:29:43 PM     LOG [RuleExecutorService] Execution of rules for 'Low rated movies' done.
maintainerr  | [Nest] 76  - 01/25/2024, 12:29:43 PM     LOG [RuleExecutorService] Synced collection 'Leaving soon' with Plex
maintainerr  | [Nest] 76  - 01/25/2024, 12:31:11 PM     LOG [RuleExecutorService] Starting Execution of all active rules.
maintainerr  | [Nest] 76  - 01/25/2024, 12:31:12 PM     LOG [RuleExecutorService] Executing rules for 'Low rated movies'
maintainerr  | [Nest] 76  - 01/25/2024, 12:31:28 PM     LOG [RuleExecutorService] Removing 2 media items from 'Leaving soon'.
maintainerr  | [Nest] 76  - 01/25/2024, 12:31:28 PM     LOG [RuleExecutorService] Adding 1 media items to 'Leaving soon'.
maintainerr  | [Nest] 76  - 01/25/2024, 12:31:28 PM     LOG [CollectionsService] Adding media with id 10862 to collection..
maintainerr  | [Nest] 76  - 01/25/2024, 12:31:28 PM     LOG [CollectionsService] Removing media with id 10711 from collection..
maintainerr  | [Nest] 76  - 01/25/2024, 12:31:28 PM     LOG [CollectionsService] Removing media with id 23259 from collection..
maintainerr  | [Nest] 76  - 01/25/2024, 12:31:28 PM     LOG [RuleExecutorService] Execution of rules for 'Low rated movies' done.
maintainerr  | [Nest] 76  - 01/25/2024, 12:31:28 PM     LOG [RuleExecutorService] Synced collection 'Leaving soon' with Plex

As a last note, I'm not sure what I did initially to make the first item (10711) stay in the collection permanently, I remember that I restarted the Docker image, but that seems to have no effect now.

Edit: I would try and help out by trying to write a test to reproduce this if it was a language I was more familiar with, but I haven't worked with JS since jQuery was popular and I've never learned TS, plus I've never used Jest, so I'm afraid I don't really know how helpful I can be.

[jorenn92]

Adding that additional rule gives some interesting insight, I think:

For the item (10711) that always stays in the collection:

...
        - operator: AND
          action: equals
          firstValueName: Plex - Present in amount of other collections
          firstValue: 1
          secondValueName: number
          secondValue: 0
          result: false
        - operator: OR
          action: equals
          firstValueName: Plex - [list] Collections media is present in (titles)
          firstValue:
            - Leaving soon
          secondValueName: text
          secondValue: foobar
          result: false

For the shifting item (10862) that is currently in the collection:

...
        - operator: AND
          action: equals
          firstValueName: Plex - Present in amount of other collections
          firstValue: 0
          secondValueName: number
          secondValue: 0
          result: true
        - operator: OR
          action: equals
          firstValueName: Plex - [list] Collections media is present in (titles)
          firstValue: null
          secondValueName: text
          secondValue: foobar
          result: false

For the shifting item (23259) that is currently not in the collection:

...
        - operator: AND
          action: equals
          firstValueName: Plex - Present in amount of other collections
          firstValue: 1
          secondValueName: number
          secondValue: 0
          result: false
        - operator: OR
          action: equals
          firstValueName: Plex - [list] Collections media is present in (titles)
          firstValue:
            - Leaving soon
          secondValueName: text
          secondValue: foobar
          result: false

You would expect that the first and second item would have the same results for that ruleset, because both are currently in the "Leaving soon" collection, but it's the one currently not in the collection that has the same results.

This brings me back to my initial idea about the cached metadata might be out of sync, which might cause this function to then return inaccurate data: https://github.com/jorenn92/Maintainerr/blob/main/server/src/modules/rules/getter/plex-getter.service.ts#L97-L111

Now, back to what you first suggested with removing that item (10711) that stayed in the collection.

When it's not in the collection (before rules have run):

...
        - operator: AND
          action: equals
          firstValueName: Plex - Present in amount of other collections
          firstValue: 0
          secondValueName: number
          secondValue: 0
          result: true
        - operator: OR
          action: equals
          firstValueName: Plex - [list] Collections media is present in (titles)
          firstValue: null
          secondValueName: text
          secondValue: foobar
          result: false

Running the rule added it to the collection (as expected), but checking it again looks exactly like before, like the cached metadata was not updated:

...
        - operator: AND
          action: equals
          firstValueName: Plex - Present in amount of other collections
          firstValue: 0
          secondValueName: number
          secondValue: 0
          result: true
        - operator: OR
          action: equals
          firstValueName: Plex - [list] Collections media is present in (titles)
          firstValue: null
          secondValueName: text
          secondValue: foobar
          result: false

Running the rule removes it from the collection (I guess this is what I expected?), and it's back to looking like the other switching item:

...
        - operator: AND
          action: equals
          firstValueName: Plex - Present in amount of other collections
          firstValue: 1
          secondValueName: number
          secondValue: 0
          result: false
        - operator: OR
          action: equals
          firstValueName: Plex - [list] Collections media is present in (titles)
          firstValue:
            - Leaving soon
          secondValueName: text
          secondValue: foobar
          result: false

Re-running the rule now adds it or removes it (and another of the switching items). Here are the logs, just for good measure:

maintainerr  | [Nest] 76  - 01/25/2024, 12:22:41 PM     LOG [RulesService] Removed exclusion for media with id 10711 and rulegroup id 9
maintainerr  | [Nest] 76  - 01/25/2024, 12:24:56 PM     LOG [RuleExecutorService] Starting Execution of all active rules.
maintainerr  | [Nest] 76  - 01/25/2024, 12:24:56 PM     LOG [RuleExecutorService] Executing rules for 'Low rated movies'
maintainerr  | [Nest] 76  - 01/25/2024, 12:25:13 PM     LOG [RuleExecutorService] Removing 1 media items from 'Leaving soon'.
maintainerr  | [Nest] 76  - 01/25/2024, 12:25:13 PM     LOG [RuleExecutorService] Adding 2 media items to 'Leaving soon'.
maintainerr  | [Nest] 76  - 01/25/2024, 12:25:14 PM     LOG [CollectionsService] Adding media with id 10711 to collection..
maintainerr  | [Nest] 76  - 01/25/2024, 12:25:14 PM     LOG [CollectionsService] Adding media with id 23259 to collection..
maintainerr  | [Nest] 76  - 01/25/2024, 12:25:14 PM     LOG [CollectionsService] Removing media with id 10862 from collection..
maintainerr  | [Nest] 76  - 01/25/2024, 12:25:14 PM     LOG [RuleExecutorService] Execution of rules for 'Low rated movies' done.
maintainerr  | [Nest] 76  - 01/25/2024, 12:25:14 PM     LOG [RuleExecutorService] Synced collection 'Leaving soon' with Plex
maintainerr  | [Nest] 76  - 01/25/2024, 12:26:51 PM     LOG [RuleExecutorService] Starting Execution of all active rules.
maintainerr  | [Nest] 76  - 01/25/2024, 12:26:51 PM     LOG [RuleExecutorService] Executing rules for 'Low rated movies'
maintainerr  | [Nest] 76  - 01/25/2024, 12:27:08 PM     LOG [RuleExecutorService] Removing 2 media items from 'Leaving soon'.
maintainerr  | [Nest] 76  - 01/25/2024, 12:27:08 PM     LOG [RuleExecutorService] Adding 1 media items to 'Leaving soon'.
maintainerr  | [Nest] 76  - 01/25/2024, 12:27:08 PM     LOG [CollectionsService] Adding media with id 10862 to collection..
maintainerr  | [Nest] 76  - 01/25/2024, 12:27:09 PM     LOG [CollectionsService] Removing media with id 10711 from collection..
maintainerr  | [Nest] 76  - 01/25/2024, 12:27:09 PM     LOG [CollectionsService] Removing media with id 23259 from collection..
maintainerr  | [Nest] 76  - 01/25/2024, 12:27:09 PM     LOG [RuleExecutorService] Execution of rules for 'Low rated movies' done.
maintainerr  | [Nest] 76  - 01/25/2024, 12:27:09 PM     LOG [RuleExecutorService] Synced collection 'Leaving soon' with Plex
maintainerr  | [Nest] 76  - 01/25/2024, 12:29:25 PM     LOG [RuleExecutorService] Starting Execution of all active rules.
maintainerr  | [Nest] 76  - 01/25/2024, 12:29:25 PM     LOG [RuleExecutorService] Executing rules for 'Low rated movies'
maintainerr  | [Nest] 76  - 01/25/2024, 12:29:42 PM     LOG [RuleExecutorService] Removing 1 media items from 'Leaving soon'.
maintainerr  | [Nest] 76  - 01/25/2024, 12:29:42 PM     LOG [RuleExecutorService] Adding 2 media items to 'Leaving soon'.
maintainerr  | [Nest] 76  - 01/25/2024, 12:29:42 PM     LOG [CollectionsService] Adding media with id 10711 to collection..
maintainerr  | [Nest] 76  - 01/25/2024, 12:29:43 PM     LOG [CollectionsService] Adding media with id 23259 to collection..
maintainerr  | [Nest] 76  - 01/25/2024, 12:29:43 PM     LOG [CollectionsService] Removing media with id 10862 from collection..
maintainerr  | [Nest] 76  - 01/25/2024, 12:29:43 PM     LOG [RuleExecutorService] Execution of rules for 'Low rated movies' done.
maintainerr  | [Nest] 76  - 01/25/2024, 12:29:43 PM     LOG [RuleExecutorService] Synced collection 'Leaving soon' with Plex
maintainerr  | [Nest] 76  - 01/25/2024, 12:31:11 PM     LOG [RuleExecutorService] Starting Execution of all active rules.
maintainerr  | [Nest] 76  - 01/25/2024, 12:31:12 PM     LOG [RuleExecutorService] Executing rules for 'Low rated movies'
maintainerr  | [Nest] 76  - 01/25/2024, 12:31:28 PM     LOG [RuleExecutorService] Removing 2 media items from 'Leaving soon'.
maintainerr  | [Nest] 76  - 01/25/2024, 12:31:28 PM     LOG [RuleExecutorService] Adding 1 media items to 'Leaving soon'.
maintainerr  | [Nest] 76  - 01/25/2024, 12:31:28 PM     LOG [CollectionsService] Adding media with id 10862 to collection..
maintainerr  | [Nest] 76  - 01/25/2024, 12:31:28 PM     LOG [CollectionsService] Removing media with id 10711 from collection..
maintainerr  | [Nest] 76  - 01/25/2024, 12:31:28 PM     LOG [CollectionsService] Removing media with id 23259 from collection..
maintainerr  | [Nest] 76  - 01/25/2024, 12:31:28 PM     LOG [RuleExecutorService] Execution of rules for 'Low rated movies' done.
maintainerr  | [Nest] 76  - 01/25/2024, 12:31:28 PM     LOG [RuleExecutorService] Synced collection 'Leaving soon' with Plex

As a last note, I'm not sure what I did initially to make the first item (10711) stay in the collection permanently, I remember that I restarted the Docker image, but that seems to have no effect now.

Edit: I would try and help out by trying to write a test to reproduce this if it was a language I was more familiar with, but I haven't worked with JS since jQuery was popular and I've never learned TS, plus I've never used Jest, so I'm afraid I don't really know how helpful I can be.

Thank you for your input! You may be on to something.. Upon further investigation, it seems that the problem may be associated with the particular stage at which I clear the cache.

Currently, I clear the cache at the initiation of the rule execution task. So, if an item is added to a collection by one ruleset, the next ruleset may utilize outdated metadata due to caching by the previous ruleset. So if the first one added it to it's collection, the second one won't see it during that run. Could this potentially explain the observed behavior?

Here is the portion of the code where the cache is cleared:

Maintainerr/server/src/modules/rules/tasks/rule-executor.service.ts

Line 85 in cd5df98

value.flush();

[jorenn92]

@bolmstedt, PR #801 might fix your issue. Once merged, you should be able to try it out with the latest develop/main image.

[bolmstedt]

@bolmstedt, PR #801 might fix your issue. Once merged, you should be able to try it out with the latest develop/main image.

Tried the latest develop image, and although it is flushing the cache, sadly the behavior looks just like before when running the rule:

maintainerr  | [maintainerr] | 26/01/2024 13:40:58  [INFO] [RuleExecutorService] Starting Execution of all active rules
maintainerr  | [maintainerr] | 26/01/2024 13:40:59  [INFO] [RuleExecutorService] Executing rules for 'Low rated movies'
maintainerr  | [maintainerr] | 26/01/2024 13:40:59  [INFO] [RulesService] Flushed Plex cache because a rule in the group required it
maintainerr  | [maintainerr] | 26/01/2024 13:41:17  [INFO] [RuleExecutorService] Removing 1 media items from 'Leaving soon'.
maintainerr  | [maintainerr] | 26/01/2024 13:41:17  [INFO] [RuleExecutorService] Adding 2 media items to 'Leaving soon'.
maintainerr  | [maintainerr] | 26/01/2024 13:41:17  [INFO] [CollectionsService] Adding media with id 10711 to collection..
maintainerr  | [maintainerr] | 26/01/2024 13:41:18  [INFO] [CollectionsService] Adding media with id 23259 to collection..
maintainerr  | [maintainerr] | 26/01/2024 13:41:18  [INFO] [CollectionsService] Removing media with id 10862 from collection..
maintainerr  | [maintainerr] | 26/01/2024 13:41:18  [INFO] [RuleExecutorService] Execution of rules for 'Low rated movies' done.
maintainerr  | [maintainerr] | 26/01/2024 13:41:18  [INFO] [RuleExecutorService] Synced collection 'Leaving soon' with Plex
maintainerr  | [maintainerr] | 26/01/2024 13:42:20  [INFO] [RuleExecutorService] Starting Execution of all active rules
maintainerr  | [maintainerr] | 26/01/2024 13:42:20  [INFO] [RuleExecutorService] Executing rules for 'Low rated movies'
maintainerr  | [maintainerr] | 26/01/2024 13:42:20  [INFO] [RulesService] Flushed Plex cache because a rule in the group required it
maintainerr  | [maintainerr] | 26/01/2024 13:42:39  [INFO] [RuleExecutorService] Removing 2 media items from 'Leaving soon'.
maintainerr  | [maintainerr] | 26/01/2024 13:42:39  [INFO] [RuleExecutorService] Adding 1 media items to 'Leaving soon'.
maintainerr  | [maintainerr] | 26/01/2024 13:42:39  [INFO] [CollectionsService] Adding media with id 10862 to collection..
maintainerr  | [maintainerr] | 26/01/2024 13:42:39  [INFO] [CollectionsService] Removing media with id 10711 from collection..
maintainerr  | [maintainerr] | 26/01/2024 13:42:39  [INFO] [CollectionsService] Removing media with id 23259 from collection..
maintainerr  | [maintainerr] | 26/01/2024 13:42:40  [INFO] [RuleExecutorService] Execution of rules for 'Low rated movies' done.
maintainerr  | [maintainerr] | 26/01/2024 13:42:40  [INFO] [RuleExecutorService] Synced collection 'Leaving soon' with Plex
maintainerr  | [maintainerr] | 26/01/2024 13:44:03  [INFO] [RuleExecutorService] Starting Execution of all active rules
maintainerr  | [maintainerr] | 26/01/2024 13:44:03  [INFO] [RuleExecutorService] Executing rules for 'Low rated movies'
maintainerr  | [maintainerr] | 26/01/2024 13:44:03  [INFO] [RulesService] Flushed Plex cache because a rule in the group required it
maintainerr  | [maintainerr] | 26/01/2024 13:44:21  [INFO] [RuleExecutorService] Removing 1 media items from 'Leaving soon'.
maintainerr  | [maintainerr] | 26/01/2024 13:44:21  [INFO] [RuleExecutorService] Adding 2 media items to 'Leaving soon'.
maintainerr  | [maintainerr] | 26/01/2024 13:44:21  [INFO] [CollectionsService] Adding media with id 10711 to collection..
maintainerr  | [maintainerr] | 26/01/2024 13:44:21  [INFO] [CollectionsService] Adding media with id 23259 to collection..
maintainerr  | [maintainerr] | 26/01/2024 13:44:22  [INFO] [CollectionsService] Removing media with id 10862 from collection..
maintainerr  | [maintainerr] | 26/01/2024 13:44:22  [INFO] [RuleExecutorService] Execution of rules for 'Low rated movies' done.
maintainerr  | [maintainerr] | 26/01/2024 13:44:22  [INFO] [RuleExecutorService] Synced collection 'Leaving soon' with Plex

Item 10862, that was just removed has this test output:

...
        - operator: AND
          action: equals
          firstValueName: Plex - Present in amount of other collections
          firstValue: 1
          secondValueName: number
          secondValue: 0
          result: false
        - operator: OR
          action: equals
          firstValueName: Plex - [list] Collections media is present in (titles)
          firstValue:
            - Leaving soon
          secondValueName: text
          secondValue: foobar
          result: false

Item 10711 that was just added has this output:

...
        - operator: AND
          action: equals
          firstValueName: Plex - Present in amount of other collections
          firstValue: 0
          secondValueName: number
          secondValue: 0
          result: true
        - operator: OR
          action: equals
          firstValueName: Plex - [list] Collections media is present in (titles)
          firstValue: null
          secondValueName: text
          secondValue: foobar
          result: false

I though that your fix would solve the issue, but I'm not sure why it seems to have no effect.

I am now wondering if there is something strange with this Plex collection getter, if it is not filtering the custom collection name properly, because I would assume that a movie in that collection would be counted as being in 0 collection, not 1.

[jorenn92]

@bolmstedt, PR #801 might fix your issue. Once merged, you should be able to try it out with the latest develop/main image.

Tried the latest develop image, and although it is flushing the cache, sadly the behavior looks just like before when running the rule:

maintainerr  | [maintainerr] | 26/01/2024 13:40:58  [INFO] [RuleExecutorService] Starting Execution of all active rules
maintainerr  | [maintainerr] | 26/01/2024 13:40:59  [INFO] [RuleExecutorService] Executing rules for 'Low rated movies'
maintainerr  | [maintainerr] | 26/01/2024 13:40:59  [INFO] [RulesService] Flushed Plex cache because a rule in the group required it
maintainerr  | [maintainerr] | 26/01/2024 13:41:17  [INFO] [RuleExecutorService] Removing 1 media items from 'Leaving soon'.
maintainerr  | [maintainerr] | 26/01/2024 13:41:17  [INFO] [RuleExecutorService] Adding 2 media items to 'Leaving soon'.
maintainerr  | [maintainerr] | 26/01/2024 13:41:17  [INFO] [CollectionsService] Adding media with id 10711 to collection..
maintainerr  | [maintainerr] | 26/01/2024 13:41:18  [INFO] [CollectionsService] Adding media with id 23259 to collection..
maintainerr  | [maintainerr] | 26/01/2024 13:41:18  [INFO] [CollectionsService] Removing media with id 10862 from collection..
maintainerr  | [maintainerr] | 26/01/2024 13:41:18  [INFO] [RuleExecutorService] Execution of rules for 'Low rated movies' done.
maintainerr  | [maintainerr] | 26/01/2024 13:41:18  [INFO] [RuleExecutorService] Synced collection 'Leaving soon' with Plex
maintainerr  | [maintainerr] | 26/01/2024 13:42:20  [INFO] [RuleExecutorService] Starting Execution of all active rules
maintainerr  | [maintainerr] | 26/01/2024 13:42:20  [INFO] [RuleExecutorService] Executing rules for 'Low rated movies'
maintainerr  | [maintainerr] | 26/01/2024 13:42:20  [INFO] [RulesService] Flushed Plex cache because a rule in the group required it
maintainerr  | [maintainerr] | 26/01/2024 13:42:39  [INFO] [RuleExecutorService] Removing 2 media items from 'Leaving soon'.
maintainerr  | [maintainerr] | 26/01/2024 13:42:39  [INFO] [RuleExecutorService] Adding 1 media items to 'Leaving soon'.
maintainerr  | [maintainerr] | 26/01/2024 13:42:39  [INFO] [CollectionsService] Adding media with id 10862 to collection..
maintainerr  | [maintainerr] | 26/01/2024 13:42:39  [INFO] [CollectionsService] Removing media with id 10711 from collection..
maintainerr  | [maintainerr] | 26/01/2024 13:42:39  [INFO] [CollectionsService] Removing media with id 23259 from collection..
maintainerr  | [maintainerr] | 26/01/2024 13:42:40  [INFO] [RuleExecutorService] Execution of rules for 'Low rated movies' done.
maintainerr  | [maintainerr] | 26/01/2024 13:42:40  [INFO] [RuleExecutorService] Synced collection 'Leaving soon' with Plex
maintainerr  | [maintainerr] | 26/01/2024 13:44:03  [INFO] [RuleExecutorService] Starting Execution of all active rules
maintainerr  | [maintainerr] | 26/01/2024 13:44:03  [INFO] [RuleExecutorService] Executing rules for 'Low rated movies'
maintainerr  | [maintainerr] | 26/01/2024 13:44:03  [INFO] [RulesService] Flushed Plex cache because a rule in the group required it
maintainerr  | [maintainerr] | 26/01/2024 13:44:21  [INFO] [RuleExecutorService] Removing 1 media items from 'Leaving soon'.
maintainerr  | [maintainerr] | 26/01/2024 13:44:21  [INFO] [RuleExecutorService] Adding 2 media items to 'Leaving soon'.
maintainerr  | [maintainerr] | 26/01/2024 13:44:21  [INFO] [CollectionsService] Adding media with id 10711 to collection..
maintainerr  | [maintainerr] | 26/01/2024 13:44:21  [INFO] [CollectionsService] Adding media with id 23259 to collection..
maintainerr  | [maintainerr] | 26/01/2024 13:44:22  [INFO] [CollectionsService] Removing media with id 10862 from collection..
maintainerr  | [maintainerr] | 26/01/2024 13:44:22  [INFO] [RuleExecutorService] Execution of rules for 'Low rated movies' done.
maintainerr  | [maintainerr] | 26/01/2024 13:44:22  [INFO] [RuleExecutorService] Synced collection 'Leaving soon' with Plex

Item 10862, that was just removed has this test output:

...
        - operator: AND
          action: equals
          firstValueName: Plex - Present in amount of other collections
          firstValue: 1
          secondValueName: number
          secondValue: 0
          result: false
        - operator: OR
          action: equals
          firstValueName: Plex - [list] Collections media is present in (titles)
          firstValue:
            - Leaving soon
          secondValueName: text
          secondValue: foobar
          result: false

Item 10711 that was just added has this output:

...
        - operator: AND
          action: equals
          firstValueName: Plex - Present in amount of other collections
          firstValue: 0
          secondValueName: number
          secondValue: 0
          result: true
        - operator: OR
          action: equals
          firstValueName: Plex - [list] Collections media is present in (titles)
          firstValue: null
          secondValueName: text
          secondValue: foobar
          result: false

I though that your fix would solve the issue, but I'm not sure why it seems to have no effect.

I am now wondering if there is something strange with this Plex collection getter, if it is not filtering the custom collection name properly, because I would assume that a movie in that collection would be counted as being in 0 collection, not 1.

Just to make sure, I'm beginning to feel a bit confused. Is the 'Leaving Soon' collection in anyway related to this rule? Is it for instance, the custom collection name?

And are both shifting items consistently included in the 'Leaving Soon' collection?

[bolmstedt]

Exactly, "Leaving soon" is the name of the custom collection related to that ruleset. All mentioned shifting items are being added and removed from that collection.

I realise now that I've not tried to not have a custom collection yet, if that makes a difference. I'll try that later tonight or tomorrow.

[jorenn92]

Exactly, "Leaving soon" is the name of the custom collection related to that ruleset. All mentioned shifting items are being added and removed from that collection.

I realise now that I've not tried to not have a custom collection yet, if that makes a difference. I'll try that later tonight or tomorrow.

You don't have to, i've figured it out. Fix in #804

As you suspected, the rule was just wrong in conjunction with a manual collection. the properties manualCollection and manualCollectionName were being accessed from ruleGroup, where they weren't available. Instead, they are present on ruleGroup.collection. This resulted in them always returning undefined, which caused the mismatch.

Additionally, I addressed another small caching oversight related to 'test media'. Previously, it would utilize the last cached metadata from Plex, which might not be up to date. Now, when testing media, it clears the item's cache first to ensure accuracy.

Once merged, you're able to test it out with the develop/main image.

[bolmstedt]

Exactly, "Leaving soon" is the name of the custom collection related to that ruleset. All mentioned shifting items are being added and removed from that collection.
I realise now that I've not tried to not have a custom collection yet, if that makes a difference. I'll try that later tonight or tomorrow.

You don't have to, i've figured it out. Fix in #804

As you suspected, the rule was just wrong in conjunction with a manual collection. the properties manualCollection and manualCollectionName were being accessed from ruleGroup, where they weren't available. Instead, they are present on ruleGroup.collection. This resulted in them always returning undefined, which caused the mismatch.

Additionally, I addressed another small caching oversight related to 'test media'. Previously, it would utilize the last cached metadata from Plex, which might not be up to date. Now, when testing media, it clears the item's cache first to ensure accuracy.

Once merged, you're able to test it out with the develop/main image.

Yes, I noticed that it started working during the weekend, thank you! I'll stay on develop until there is a new release.