Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve performance on ManyToMany chained filters #977

Closed
wants to merge 7 commits into from

Conversation

bruno-fs
Copy link

@bruno-fs bruno-fs commented Sep 20, 2018

I was experiencing the issue @eduardocalil reported on #745. At the time, the fix proposed by him wasnt accepted because:

  1. it broke a test.
  2. because the problem is more a django ORM issue than django-filter.

I can't comment on 2, but at least 1 dont block this PR anymore. The error was caused by currently unsupported versions of DRF.

@bruno-fs bruno-fs changed the title yet another attempt to fix #745 improve performance on m2m chained filters Sep 21, 2018
@bruno-fs bruno-fs changed the title improve performance on m2m chained filters Improve performance on ManyToMany chained filters Sep 21, 2018
@bruno-fs
Copy link
Author

After inspecting the tests I realized they are not actually running.

@rpkilby
Copy link
Collaborator

rpkilby commented Sep 21, 2018

After inspecting the tests I realized they are not actually running.

Can you point out which tests aren't running?

@carltongibson
Copy link
Owner

Hi.

This isn't a performance issue. The issue is that over M2M relations, this:

Blog.objects.filter(entry__headline__contains='Lennon', entry__pub_date__year=2008)

is not semantically equivalent to this:

Blog.objects.filter(entry__headline__contains='Lennon').filter(entry__pub_date__year=2008)

as documented in the Queries topic doc.

The solution here is NOT to get involved in ORM Dark Magic™.

Rather, you need to be creating a (multi-)filter that takes the (e.g. two) parameters and applies them in a single step.

The addition to Django Filter for this is:

  • Document the pattern
  • Add a filter class that allows declarative declaration.

I'll leave this open for now as a reminder.

@bruno-fs
Copy link
Author

@rpkilby about the tests being skipped, it seems that all of them were skipped 😱
this happened not only on this PR but also on master.

@bruno-fs
Copy link
Author

bruno-fs commented Sep 21, 2018

@rpkilby this is the change to make detox run and detect that I was breaking lots of tests

@rpkilby
Copy link
Collaborator

rpkilby commented Sep 21, 2018

Yikes. This looks like a huge bug in tox/detox. Those exceptions should not result in passing builds.

@carltongibson - might be a good reason to enable the coverage status check?

@rpkilby
Copy link
Collaborator

rpkilby commented Sep 21, 2018

I would recommend that we ditch detox in favor of regular tox. Honestly, I don't think we get much from parallelizing tox. The tests are pretty fast - most of the build time is just setting up the Travis container.

@carltongibson
Copy link
Owner

@rpkilby Are you good to make the PR for that?

@rpkilby
Copy link
Collaborator

rpkilby commented Sep 24, 2018

Sure.

@rpkilby
Copy link
Collaborator

rpkilby commented Nov 1, 2018

lol I never got around to making that PR. Done.

@rpkilby
Copy link
Collaborator

rpkilby commented Nov 1, 2018

@carltongibson - I think you can close this now given the PR.

@carltongibson
Copy link
Owner

I left this open to document the pattern and/or add a multi-parameter filter base class or such.
(Have had ===0 time to look at that. 🙄)

@rpkilby
Copy link
Collaborator

rpkilby commented Nov 1, 2018

Ah - thought you meant about the tox/detox thing. 👍

@carltongibson
Copy link
Owner

Moved documentation issue into #1020

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants