Unable to run a cron every 5 seconds, when job_id is set? (does the minimum have to be 60 seconds?)

[epicwhale]

I need to run a cron in my worker every 5 seconds, but I also want to provide it a job_id so there's no duplicates running in parallel. The next cron task for the job should only run if the previous one is completed.

My cron definition:

    cron(
        my_task,
        unique=True,
        job_id="cron_my_task",  # to prevent duplicate task runs
        second=set(range(0, 60, 5)),         # every 5 seconds
        keep_result=0, 
        max_tries=1,
    ),

But when I configure a worker with this, I notice there's an in-progress entry created with a TTL of 60 seconds, which prevents any future cron runs from running till it expires, even if my tasks is completed in a few seconds.

Am I missing something important here? As this behavior feels rather unpredictable, and undocumented.

Digging deeper into the code, I see there's a
keep_cronjob_progress = 60 <--- # how long to keep the "in_progress" key after a cron job ends to prevent the job duplication which is a constant that can't be modified:

arq/arq/constants.py

Line 12 in 1315583

keep_cronjob_progress = 60

What's the recommended way to accomplish my requirement of running a task every 5 seconds, provided the previous cron run of that task is completed?