-
Notifications
You must be signed in to change notification settings - Fork 755
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Filtering is reaalllllllly slow #1143
Comments
You have a model choice field or similar trying to render many thousands of rows. |
Could we have a global setting that just renders those as integer fields? |
No. Use prefetch_related and co. (As per usual.) (Or customise the widget if that's what you prefer.) |
The issue isn't the query speed, it's serving and rendering the dropdown lists, especially when combined with the debug toolbar. At a certain point, the REST api browsable interface just times out test servers due ram overusage.
Changing the widget to a NumberInput works fine, but not many people know how to set it up. Here's the change for ForeignKeys:
I haven't tried a ModelMultipleChoiceFilter yet, but I imagine you can do something similar. It might be good to add something about this to the ModelChoiceFilter documentation. For regular django projects, people will need to either filter the queryset or switch to a more complex widget, but django rest api projects really just need something to fix the browsable test interface. |
@jonathan-golorry thanks for the example! For our internal stuff we landed on making this default behavior for all foreign key fields. This way we don't have to setup classes and inherit stuff all over the place. We did this by creating a class based on your example and using it as the base class for the back end. This way you can use Settings: REST_FRAMEWORK = {
# ...
'DEFAULT_FILTER_BACKENDS': [
'someproject.filters.SomeProjectFilterBackend',
'rest_framework.filters.OrderingFilter',
'rest_framework.filters.SearchFilter',
],
# ...
from django import forms
from django.db import models
from django_filters.filterset import remote_queryset
from django_filters.rest_framework import ModelChoiceFilter
from django_filters.rest_framework.backends import DjangoFilterBackend
from django_filters.rest_framework.filterset import FilterSet
class ForeignKeyFilterSet(FilterSet):
"""
Make sure ForeignKey fields show as text input when using the API browser.
The default widget is a select and with large sets it will take a long time
to render and probably crash your browser when you try to open the select
with tens of thousands of items.
Usage:
```
from someproject.filters import ForeignKeyFilterSet
from rest_framework import viewsets
class SomeModelFilter(ForeignKeyFilterSet):
class Meta(ForeignKeyFilterSet.Meta):
fields = ['some_foreignkey']
model = SomeModel
class SomeModelViewSet(viewsets.ModelViewSet):
filterset_class = SomeModelFilter
queryset = SomeModel.objects.all()
serializer_class = serializers.SomeModelSerializer
```
If this is global via backend and you want default behavior, you need to create
a filter and use filterset_class instead of filterset_fields on your viewset(s).
```
from django_filters import rest_framework as filters
class SomeModelFilter(filters.FilterSet):
class Meta():
fields = ['some_foreignkey']
model = SomeModel
class SomeModelViewSet(viewsets.ModelViewSet):
filterset_class = SomeModelFilter
queryset = SomeModel.objects.all()
serializer_class = serializers.SomeModelSerializer
```
"""
class Meta():
filter_overrides = {
models.ForeignKey: {
'filter_class': ModelChoiceFilter,
'extra': lambda f: {
'queryset': remote_queryset(f),
'widget': forms.NumberInput,
},
},
}
class SomeProjectFilterBackend(DjangoFilterBackend):
"""
REST backend for filtering.
Use our custom ForeignKeyFilterSet so that you don't have to rename
ForeignKey fields (e.g., "model__id") or do per-filter/viewset
customizations to get the equivalent of "raw_id_fields" for
filters in the browsable API.
See ForeignKeyFilterSet to restore default behavior as needed.
"""
filterset_base = ForeignKeyFilterSet |
Also, FWIW, It'd be great if there were some equivalent to Django's |
I noticed slowness during template rendering like this and I was able to work around it. In my case, I am filter users by pk and the field is hidden, but you could just as easily enter a name (
|
Hi and thanks for this awesome library !
I am currently using it in a project and facing a huge problem : adding django-filter multiply render time by 6.
I went from 700 ms to 5 to 6 seconds. This is really huge.
I computed the server side render time which is only 0.23s. Measuring requests using django-toolbar makes an honnest 42ms request time.
This only concerns templates as soon as I remove the filter form, it's working fine again. This is something related to form rendering. I tried with and without crispy : it is the same.
I don't know what to do to improve this load time that is not acceptable.
Thanks for your help :)
The text was updated successfully, but these errors were encountered: