Document Permissions being affected by the relational API Limit

[NecoHorne]

Describe the Bug

Permissions O2M / M2O checks are limited to 100.
I have a role, the role has a O2M relation from the directus_users table to customers.

In the permissions I use a custom permission on the read field of the customer table: So the role can see anything they have created and any customers that have been assigned to them

{
  "_and": [
    {
      "_or": [
        {
          "user_created": {
            "_eq": "$CURRENT_USER"
          }
        },
        {
          "id": {
            "_in": [
              "$CURRENT_USER.customers"
            ]
          }
        }
      ]
    }
  ]
}

This works as expected except when a user has more than 100 customers assigned to them, I think the permissions check is hitting the internal API limit of 100 and just returning the first 100 customers in the data studio and API, this has a knock on effect for other items related to the customers like statements where the permissions do a similar check on customer and only returns the statements of the first 100 customers.

Double checked in MySQL to confirm the issue.

SELECT COUNT(id) AS numberOfCustomers FROM customer where account_manager = 'e1ef5c49-942e-451d-a1f0-89f33ec6258c';

returns 250 but in the API and data studio it only shows the first 100 customers.

under the role:

I have not found a way to remove the limit from the permissions using filter rules other than changing the QUERY_LIMIT_DEFAULT on the ENV file.

Hosting is self hosted, Ubuntu instance with MySQL, tested on older versions of Directus and the latest version (v10.10.5)

To Reproduce

O2M relationship between users and a table where the user has more than 100 items assigned to them, then configure permissions for a role according to that relationship.

Directus Version

10.10.5

Hosting Strategy

Self-Hosted (Custom)

[br41nslug]

I have not found a way to remove the limit from the permissions using filter rules other than changing the QUERY_LIMIT_DEFAULT on the ENV file.

As you've found you can change this default limit using the QUERY_LIMIT_DEFAULT environment variable or by providing a limit (https://docs.directus.io/reference/query.html#limit) for either on the main collection or deep->_limit (https://docs.directus.io/reference/query.html#deep) for relations as long as it is smaller than the QUERY_LIMIT_MAX.

The App itself should paginate where possible, Is the issue here that pagination is missing somewhere?

Closing the issue for now as the limit itself is by design.

[NecoHorne]

Hi @br41nslug

The issue is in the permissions checks themselves for the role. I know how to change the limit when using the API

api/src/services/permissions/index.ts extends ItemsService, which is bound by the default query limit, for permissions should this not check all instead of being limited? if it was by design I will just change the QUERY_LIMIT_DEFAULT variable. I understand the limit for the API but for custom permissions it should probably check all or document somewhere that it will only check the first default query limit of relations.

I will try out the deep limits on the permissions, thanks for responding

[br41nslug]

Ah hate to say it but that second rule ID is on of [<large array>] (*note the nested array) wont work either way 😬

So the core issue is relational values gotten via dynamic variables $current_user.something is hitting the default limit 🤔 I am torn between that being intentional or not as loading in 100+ item for each permission check aint great for performance.

[NecoHorne]

I increased the QUERY_LIMIT_DEFAULT to 250 and gives the expected results based on the second rule.

[br41nslug]

I increased the QUERY_LIMIT_DEFAULT to 250 and gives the expected results based on the second rule.

Interesting, maybe that only applies to m2m relations then, fascinating 🤔

[NecoHorne]

I am torn between that being intentional or not as loading in 100+ item for each permission check aint great for performance.

I thought this might have been an intentional design choice based on performance but was not sure, I could not find it documented somewhere that permissions would check based on the default query limit.

[NecoHorne]

I increased the QUERY_LIMIT_DEFAULT to 250 and gives the expected results based on the second rule.

Interesting, maybe that only applies to m2m relations then, fascinating 🤔

for M2M relations I have found this to work for nested arrays

{
  "_and": [
    {
      "_or": [
        {
          "customer": {
            "id": {
              "_in": [
                "$CURRENT_USER.customers"
              ]
            }
          }
        },
        {
          "customer": {
            "id": {
              "_in": [
                "$CURRENT_USER.companies.customer_id"
              ]
            }
          }
        }
      ]
    }
  ]
}

It hits the same default query limit but these arrays are much smaller as a customer might only be part of like 10ish companies max so the limit is not an issue for me here

[br41nslug]

Fascinating, that is however a separate issue i was debugging before the weekend. Am kinda wondering why that works for ya but let's keep this one on topic 😄

[br41nslug]

It hits the same default query limit but these arrays are much smaller as a customer might only be part of like 10ish companies max so the limit is not an issue for me here

Yeah the 100 default was chosen as a "sensible default" (same as the max nesting limit) so im leaning towards this being intentional and needed some docs on the matter to make sure people know about it 🤔

[NecoHorne]

Thanks for responding,

I think it might need to be mentioned somewhere here configure-custom-permissions so others don't fall into the same trap, I first thought it was my permissions but if they were wrong the user would have seen nothing and not 100 items so then I looked at the source code of the Permissions Service I saw it extended the ItemService so figured it was hitting default query limit.

[br41nslug]

Re-opening as issue for documentation to make sure it doesnt get forgotten 😄

[NecoHorne]

@br41nslug much appreciated, if you would like to discuss my M2M permissions setup for your other debugging issue I would be happy to send you the schema on discord DM if it helps out your debugging for M2M permissions.