Make short task time limit configureable #856
Conversation
|
Thanks a lot for this PR! Read the discussion on #852 , I think a default value has to be there (on the place that the celery command is run) to act as a fail safe, in case a task runs that takes too much time. Plus you get to override this default setting on any specific task you want (cannot find the doc now but it would be easy to make a test, eg run the celery process with a default of 10seconds, run a |
|
@mgogoulos the keyword argument in the supervisord command takes precedence over anything configured in python. This timeout doesn't work for me. It really depends on the speed of the hard drive too and how much throughput and IOPS is available. 300 seconds is just too short for me per chunk. Each chunk for me takes anywhere from 250-380 seconds, and as you can imagine, many would timeout with the current hard coded timeout. The order of precedence is keyword argument I verified this by setting the timeout in my actual task decorator and it was not respected and it kept using the timeout defined in supervisord. I would definitely delete the command line argument and rely strictly on the task decorator setting. |
|
For those of us not running under Docker, could you also update the SystemD unit files, please? |
Description
The short celery task timeout is hard coded in the daemon and also in the task definition. This should just use the newly introduced setting that can be configured and overwritten with the local_settings.py file
Steps
Pre-deploy
Post-deploy
@mgogoulos