Celery Beat stuck when adding a new periodic task

[Nastaszja]

Summary:

My app is growing and is now running more than 56K periodic tasks. I am using Django Celery Beat for this. I noticed that If I add a new periodic task, my celery beat is stuck.

• Celery Version: 5.1.2
• Celery-Beat Version: 2.2.1

Exact steps to reproduce the issue:

1. Start celery beat with the +56K periodic tasks
2. Add a new periodic task
3. DatabaseScheduler is stuck for hours

Detailed information

When I add a periodic task, all the stored ones stop for a long time. Looking at the celery beat logs I realized that for several hours, my celery beat was stuck at this message :

[2021-11-04 11:46:31,548: INFO/MainProcess] DatabaseScheduler: Schedule changed.
[2021-11-04 11:46:31,702: INFO/MainProcess] DatabaseScheduler: Schedule changed.
[2021-11-04 11:46:31,849: INFO/MainProcess] DatabaseScheduler: Schedule changed.
[2021-11-04 11:46:32,383: INFO/MainProcess] DatabaseScheduler: Schedule changed.
[2021-11-04 11:46:32,529: INFO/MainProcess] DatabaseScheduler: Schedule changed.
[2021-11-04 11:46:32,690: INFO/MainProcess] DatabaseScheduler: Schedule changed.
[2021-11-04 11:46:32,814: INFO/MainProcess] DatabaseScheduler: Schedule changed.
[2021-11-04 11:46:32,864: INFO/MainProcess] DatabaseScheduler: Schedule changed.
[2021-11-04 11:46:33,004: INFO/MainProcess] DatabaseScheduler: Schedule changed.
[2021-11-04 11:46:33,145: INFO/MainProcess] DatabaseScheduler: Schedule changed.
[2021-11-04 11:46:33,282: INFO/MainProcess] DatabaseScheduler: Schedule changed.
.....

It's been 3 hours like this. Is there a maximum number of periodic tasks ? How can I handle this?

[nmenoni]

Is this issue still present? Any update? Thanks