when trying to access Periodic Task add/change view - throws exception: TypeError: cannot unpack non-iterable function object

[mrchameleon]

Hello, I am using django 3.2 and django-celery-beat==2.2.0

when I go into my celery beat/ period tasks in the admin, and try to change or add a new one, I immediately get an internal server error. The stack trace is below. The add/change action works in my non-production environments and I am hosting on heroku, which probably has something to do with the issue, since in development, qa, and staging this is working fine. (and was working previously in prod some time ago...)

This stack trace repeats the same block over and over again in the log when i try the action, so it seems like some kind of recursive error

2021-12-15T16:45:46.839207+00:00 app[web.1]: During handling of the above exception, another exception occurred:
2021-12-15T16:45:46.839208+00:00 app[web.1]:
2021-12-15T16:45:46.839208+00:00 app[web.1]: Traceback (most recent call last):
2021-12-15T16:45:46.839208+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.7/site-packages/django/core/handlers/exception.py", line 47, in inner
2021-12-15T16:45:46.839208+00:00 app[web.1]: response = get_response(request)
2021-12-15T16:45:46.839208+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.7/site-packages/django/utils/deprecation.py", line 117, in __call__
2021-12-15T16:45:46.839209+00:00 app[web.1]: response = response or self.get_response(request)
2021-12-15T16:45:46.839209+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.7/site-packages/newrelic/hooks/framework_django.py", line 1183, in _wrapper
2021-12-15T16:45:46.839209+00:00 app[web.1]: return wrapped(*args, **kwargs)
2021-12-15T16:45:46.839212+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.7/site-packages/django/core/handlers/exception.py", line 49, in inner
2021-12-15T16:45:46.839212+00:00 app[web.1]: response = response_for_exception(request, exc)
2021-12-15T16:45:46.839212+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.7/site-packages/django/core/handlers/exception.py", line 114, in response_for_exception
2021-12-15T16:45:46.839212+00:00 app[web.1]: response = handle_uncaught_exception(request, get_resolver(get_urlconf()), sys.exc_info())
2021-12-15T16:45:46.839213+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.7/site-packages/newrelic/hooks/framework_django.py", line 485, in wrapper
2021-12-15T16:45:46.839213+00:00 app[web.1]: return _wrapped(*args, **kwargs)
2021-12-15T16:45:46.839213+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.7/site-packages/newrelic/hooks/framework_django.py", line 478, in _wrapped
2021-12-15T16:45:46.839213+00:00 app[web.1]: return wrapped(request, resolver, exc_info)
2021-12-15T16:45:46.839214+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.7/site-packages/django/core/handlers/exception.py", line 152, in handle_uncaught_exception
2021-12-15T16:45:46.839214+00:00 app[web.1]: callback = resolver.resolve_error_handler(500)
2021-12-15T16:45:46.839214+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.7/site-packages/newrelic/hooks/framework_django.py", line 621, in wrapper
2021-12-15T16:45:46.839214+00:00 app[web.1]: callback, param_dict = wrapped(*args, **kwargs)
2021-12-15T16:45:46.839214+00:00 app[web.1]: TypeError: cannot unpack non-iterable function object
2021-12-15T16:45:46.839215+00:00 app[web.1]:
2021-12-15T16:45:46.839215+00:00 app[web.1]: During handling of the above exception, another exception occurred:
2021-12-15T16:45:46.839215+00:00 app[web.1]:
2021-12-15T16:45:46.839215+00:00 app[web.1]: Traceback (most recent call last):
2021-12-15T16:45:46.839216+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.7/site-packages/gunicorn/workers/sync.py", line 135, in handle
2021-12-15T16:45:46.839216+00:00 app[web.1]: self.handle_request(listener, req, client, addr)
2021-12-15T16:45:46.839216+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.7/site-packages/gunicorn/workers/sync.py", line 176, in handle_request
2021-12-15T16:45:46.839216+00:00 app[web.1]: respiter = self.wsgi(environ, resp.start_response)
2021-12-15T16:45:46.839216+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.7/site-packages/newrelic/api/wsgi_application.py", line 665, in _nr_wsgi_application_wrapper_
2021-12-15T16:45:46.839217+00:00 app[web.1]: environ, _start_response, transaction)
2021-12-15T16:45:46.839217+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.7/site-packages/newrelic/api/wsgi_application.py", line 193, in __init__
2021-12-15T16:45:46.839217+00:00 app[web.1]: self.start_response)
2021-12-15T16:45:46.839218+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.7/site-packages/newrelic/api/wsgi_application.py", line 555, in _nr_wsgi_application_wrapper_
2021-12-15T16:45:46.839218+00:00 app[web.1]: return wrapped(*args, **kwargs)
2021-12-15T16:45:46.839218+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.7/site-packages/django/core/handlers/wsgi.py", line 133, in __call__
2021-12-15T16:45:46.839219+00:00 app[web.1]: response = self.get_response(request)
2021-12-15T16:45:46.839219+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.7/site-packages/newrelic/hooks/framework_django.py", line 418, in _nr_wrapper_BaseHandler_get_response_
2021-12-15T16:45:46.839219+00:00 app[web.1]: response = wrapped(*args, **kwargs)
2021-12-15T16:45:46.839219+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.7/site-packages/django/core/handlers/base.py", line 130, in get_response
2021-12-15T16:45:46.839220+00:00 app[web.1]: response = self._middleware_chain(request)
2021-12-15T16:45:46.839220+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.7/site-packages/newrelic/hooks/framework_django.py", line 1183, in _wrapper
2021-12-15T16:45:46.839220+00:00 app[web.1]: return wrapped(*args, **kwargs)
2021-12-15T16:45:46.839220+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.7/site-packages/django/core/handlers/exception.py", line 49, in inner
2021-12-15T16:45:46.839220+00:00 app[web.1]: response = response_for_exception(request, exc)
2021-12-15T16:45:46.839221+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.7/site-packages/django/core/handlers/exception.py", line 114, in response_for_exception
2021-12-15T16:45:46.839221+00:00 app[web.1]: response = handle_uncaught_exception(request, get_resolver(get_urlconf()), sys.exc_info())
2021-12-15T16:45:46.839223+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.7/site-packages/newrelic/hooks/framework_django.py", line 485, in wrapper
2021-12-15T16:45:46.839223+00:00 app[web.1]: return _wrapped(*args, **kwargs)
2021-12-15T16:45:46.839223+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.7/site-packages/newrelic/hooks/framework_django.py", line 478, in _wrapped
2021-12-15T16:45:46.839223+00:00 app[web.1]: return wrapped(request, resolver, exc_info)
2021-12-15T16:45:46.839224+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.7/site-packages/django/core/handlers/exception.py", line 152, in handle_uncaught_exception
2021-12-15T16:45:46.839224+00:00 app[web.1]: callback = resolver.resolve_error_handler(500)
2021-12-15T16:45:46.839224+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.7/site-packages/newrelic/hooks/framework_django.py", line 621, in wrapper
2021-12-15T16:45:46.839224+00:00 app[web.1]: callback, param_dict = wrapped(*args, **kwargs)
2021-12-15T16:45:46.839225+00:00 app[web.1]: TypeError: cannot unpack non-iterable function object

[mrchameleon]

updated stack trace to indicate what may be causing the non-iterable function object error. with more information pointing at some kind of issue with the form rendering... ?

response = wrapped_callback(request, *callback_args, **callback_kwargs)
File /app/.heroku/python/lib/python3.7/site-packages/newrelic/hooks/framework_django.py line 539 in wrapper
return wrapped(*args, **kwargs)
File /app/.heroku/python/lib/python3.7/site-packages/django/contrib/admin/options.py line 616 in wrapper
return self.admin_site.admin_view(view)(*args, **kwargs)
File /app/.heroku/python/lib/python3.7/site-packages/django/utils/decorators.py line 130 in _wrapped_view
response = view_func(request, *args, **kwargs)
File /app/.heroku/python/lib/python3.7/site-packages/django/views/decorators/cache.py line 44 in _wrapped_view_func
response = view_func(request, *args, **kwargs)
File /app/.heroku/python/lib/python3.7/site-packages/django/contrib/admin/sites.py line 232 in inner
return view(request, *args, **kwargs)
File /app/.heroku/python/lib/python3.7/site-packages/django/contrib/admin/options.py line 1655 in add_view
return self.changeform_view(request, None, form_url, extra_context)
File /app/.heroku/python/lib/python3.7/site-packages/django/utils/decorators.py line 43 in _wrapper
return bound_method(*args, **kwargs)
File /app/.heroku/python/lib/python3.7/site-packages/django/utils/decorators.py line 130 in _wrapped_view
response = view_func(request, *args, **kwargs)
File /app/.heroku/python/lib/python3.7/site-packages/django/contrib/admin/options.py line 1538 in changeform_view
return self._changeform_view(request, object_id, form_url, extra_context)
File /app/.heroku/python/lib/python3.7/site-packages/django/contrib/admin/options.py line 1598 in _changeform_view
form = ModelForm(initial=initial)
File /app/.heroku/python/lib/python3.7/site-packages/django/forms/models.py line 313 in __init__
renderer=renderer,
File /app/.heroku/python/lib/python3.7/site-packages/django/forms/forms.py line 85 in __init__
self.fields = copy.deepcopy(self.base_fields)
File /app/.heroku/python/lib/python3.7/copy.py line 150 in deepcopy
y = copier(x, memo)
File /app/.heroku/python/lib/python3.7/copy.py line 241 in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File /app/.heroku/python/lib/python3.7/copy.py line 161 in deepcopy
y = copier(memo)
File /app/.heroku/python/lib/python3.7/site-packages/django/forms/fields.py line 776 in __deepcopy__
result = super().__deepcopy__(memo)
File /app/.heroku/python/lib/python3.7/site-packages/django/forms/fields.py line 204 in __deepcopy__
result.widget = copy.deepcopy(self.widget, memo)
File /app/.heroku/python/lib/python3.7/copy.py line 161 in deepcopy
y = copier(memo)
File /app/.heroku/python/lib/python3.7/site-packages/django/forms/widgets.py line 570 in __deepcopy__
obj.choices = copy.copy(self.choices)
File /app/.heroku/python/lib/python3.7/site-packages/django_celery_beat/admin.py line 37 in choices
self._choices = self.tasks_as_choices()
File /app/.heroku/python/lib/python3.7/site-packages/django_celery_beat/admin.py line 29 in tasks_as_choices
_ = self._modules # noqa
File /app/.heroku/python/lib/python3.7/site-packages/kombu/utils/objects.py line 44 in __get__
value = obj.__dict__[self.__name__] = self.__get(obj)
File /app/.heroku/python/lib/python3.7/site-packages/django_celery_beat/admin.py line 48 in _modules
self.celery_app.loader.import_default_modules()
File /app/.heroku/python/lib/python3.7/site-packages/celery/loaders/base.py line 108 in import_default_modules
raise response
File /app/.heroku/python/lib/python3.7/site-packages/celery/utils/dispatch/signal.py line 288 in send
response = receiver(signal=self, sender=sender, **named)
File /app/.heroku/python/lib/python3.7/site-packages/celery/fixups/django.py line 84 in on_import_modules
self.worker_fixup.validate_models()
File /app/.heroku/python/lib/python3.7/site-packages/celery/fixups/django.py line 123 in validate_models
run_checks()
File /app/.heroku/python/lib/python3.7/site-packages/django/core/checks/registry.py line 76 in run_checks
new_errors = check(app_configs=app_configs, databases=databases)
File /app/.heroku/python/lib/python3.7/site-packages/django/core/checks/urls.py line 13 in check_url_config
return check_resolver(resolver)
File /app/.heroku/python/lib/python3.7/site-packages/django/core/checks/urls.py line 23 in check_resolver
return check_method()
File /app/.heroku/python/lib/python3.7/site-packages/django/urls/resolvers.py line 413 in check
messages.extend(check_resolver(pattern))
File /app/.heroku/python/lib/python3.7/site-packages/django/core/checks/urls.py line 23 in check_resolver
return check_method()
File /app/.heroku/python/lib/python3.7/site-packages/django/urls/resolvers.py line 414 in check
messages.extend(self._check_custom_error_handlers())
File /app/.heroku/python/lib/python3.7/site-packages/django/urls/resolvers.py line 423 in _check_custom_error_handlers
handler = self.resolve_error_handler(status_code)
File /app/.heroku/python/lib/python3.7/site-packages/newrelic/hooks/framework_django.py line 621 in wrapper
callback, param_dict = wrapped(*args, **kwargs)```

[auvipy]

I can't see any django-celery-beat related stack trace here, it says django itself