Feature request: concurrency to queue all jobs waiting on a group

[Shereef]

Current behavior

Job A arrives, finds nothing in its concurrency group and runs
Job B arrives, finds job A running in its concurrency group, looks for anyone else waiting (with the intent to kill them), finds nothing, and waits.
Job C arrives, finds job A running in its concurrency group looks for anyone else waiting (with the intent to kill them), finds job B, kills it, and waits.
Job A finishes, job C is started.

Desired behavior

• Job A arrives, finds nothing in its concurrency group and runs
• Job B arrives, finds job A running in its concurrency group, and waits.
• Job C arrives, finds job A running in its concurrency group, and waits.
• ....etc all jobs just wait in line

Sorry if this was discussed before I tried to search but didn't find anything.

Our current implementation

To give it more context:

• We have 5 team environments
• PRs push to the author's team env
• After merge main has a github action to push to all team environments
• the groups are unique to the team env it pushes to
  • We can't deploy to the same env from 2 github actions that's why concurrency is based on envs

Imagine this scenario:
We have 5 PRs for Team A
We merge to main and update the 5 PRs
All 5 will want to deploy to the env + the main branch deploy that is ongoing
the last PR to merge is going to be queued
all other PRs will be cancelled and have to be retried manually

Suggested solutions

one of the below would work

• cancel-in-queue: false a new command to disable cancelling queuing jobs
• cancel-in-progress: false would not cancel anything and leave everything in queue

[ethomson]

Hi @Shereef - thanks much for the feedback. Our initial design was focused on deployment scenarios, but we want to address the use case that you've outlined. I appreciate you outlining this so nicely and providing these suggestions. I hope that this is an area that we can address in the future.

Thanks again.

[Shereef]

@ethomson I see you left Github. Who do I ask about this task can you tag someone ?

[Tarasovych]

Any updates on this?

[miguelb-gk]

Any updates on this?

[finolacahill]

@chrispat @tuves
Is anyone following this highly desirable feature since ethomson left Github?

[finolacahill]

@chrispat @tuves Once again asking if anyone is following up on this feature since ethomson left Github?

[JimiC]

The desired behavior here, I believe, would be for jobs with the same run_id to be canceled (either in_progres or in_queue), whenever a new incremental run_id is present on the same ref.

[justin-pierce]

The OP is asking for jobs/workflows to wait and execute sequentially instead of canceling. Canceling existing workflows when a new workflow is triggered is already supported:

https://docs.github.com/en/enterprise-server@3.3/actions/using-jobs/using-concurrency

When a concurrent job or workflow is queued, if another job or workflow using the same concurrency group in the repository is in progress, the queued job or workflow will be pending. Any previously pending job or workflow in the concurrency group will be canceled. To also cancel any currently running job or workflow in the same concurrency group, specify cancel-in-progress: true.

If your issue is specific to manually rerunning the same workflow, seems like a separate issue.

[JimiC]

The said behavior is problematic as it's canceling it's own jobs in the same run. I have 6 jobs to run in each commit and they cancel each other in the same run.

[justin-pierce]

Can your issue be solved by using concurrency on the workflow level instead of the job level? Afaik concurrency on the job level is meant more for branching paths (like their fallback value example). Sorry if I'm misunderstanding your usecase though

[JimiC]

Any previously pending **job** or workflow in the concurrency group will be canceled this is what's problematic.
And because jobs of the same commit share the same group they get canceled.

[nelson6e65]

I think I just need the same, but add to a queue instead of cancel the workflow. I meant, don't run un parallel but in serial instead

This will help, for example, use the same build result or cache in multiple runs for the same commit. The first one will create the artifacts/cache and the next one will be faster

[JimiC]

That will complicate things even further.

[justin-pierce]

Was surprised to see that canceling previous runs was the only option officially supported -- waiting and executing sequentially is what I need and seems to me like a clear usecase (canceling a deployment could result in a broken state)

So far I've found this Action: https://github.com/ahmadnassri/action-workflow-queue

But ideally, workflows sitting waiting for previous workflows to finish wouldn't consume action minutes in an official feature.

[JimiC]

What I needed was exactly what https://github.com/styfle/cancel-workflow-action provided

[mro-pbraswell]

We implemented something similar with a run step, but that means an entire self-hosted runner is occupied while it's waiting for earlier workflow runs to finish.

It's disappointing that the choice is between scaling up our self-hosted runners or surpassing our build minutes quota for github-hosted runners or building a separate queue outside of github actions to handle this kind of functionality.

[duro]

We have a long running workflow that goes through several jobs to deploy AWS infrastructure. If one of them was cancelled mid-way, or deploys happened on to of one another, this would cause some pretty serious issues.

We definitely need a way to queue jobs in sequence without consuming action minutes

[koreyGambill]

Brainstorming - Eventually I will probably have to implement a specific self-hosted group. Maybe like 25 runners on a t3.small. Their job would be to handle queues. So I'd have a workflow or job dedicated to queing, and my larger runners (c7a.xlarge) wouldn't have to sit and wait for 15 mins at a time - instead I can have a single t3.small for all repos.

[liry]

@justin-pierce the action-workflow-queue has an issue when there are multiple workflows started in the same second. See #45.

@koreyGambill exactly, but do you know about some simple yet reliable queuing solution for that?

[kmorkans-lumoa]

Would be great to be able to queue workflow runs (ideally without consuming action minutes). Was surprised that new runs cancel existing ones by default.

[ashwinv11]

We hit this limitation while testing out the Merge Queue feature that is currently in limited beta. Our use case is ensuring that only one workflow is running for the PR at the top of the merge queue. When concurrency is set at the workflow level (for the merge_group event type) and multiple PRs are queued within the merge queue, the system cancels queued check suites once a new PR is added to the queue. It would be fantastic if we had the ability to disable canceling queued jobs!

[akagr]

If you're using self-hosted runners, a workaround/hack we used was to label one of these runners specifically, and then make all the jobs that need to be in a concurrency group just run on this labelled runner instead. No concurrency shenanigans. That causes them to queue up globally across all workflows without any caps on number of waiting jobs.

The only major limitation is that if this runner fails, these jobs won't use any other runner. That was considered an acceptable risk to unblock us until we had a better solution.

[liry]

Well, this is possible only for cases, where you need to serialize only a single job.
Once you have a workflow where multiple jobs need to finish in the given order before new workflow is started (e.g. job-A: deploy to test env, job-B: run tests against the env), then it's not very well usable.

[AvivPeledTalon]

My team also needs this feature.

[clamattia]

My team also needs this.

[MitchellPaff]

My team also need this feature - mainly for queueing mulitple actions each with Terraform plan/deploy steps which cannot be run at same time due to a lock on terraform state.

[galz-cyera]

A must have for my team

[yershalom]

Really hard to run terrafrom/terragrunt at scale without it

[badbort]

Really needing this to be implemented.

I am currently using https://github.com/ben-z/gh-action-mutex for now, but this does pollute the git history.

[dumganhar]

Need this feature too.

[bob-ashforth-sp]

GitHub, I'll add my team's need for this feature as well. I see a LONG list of replies like this, with no response from GitHub. (I'm in DevOps.)

I'm going to put in a GitHub support ticket referencing this request to see if there are any plans to implement it.

Our use cases is that I have created a highly parallelized building and deploying four distinct services in a monorepo. In this workflow, the update_service action is called by the jobs for each of those services, AND we have a very active CI/CD environment which can be processing several merges almost simultaneously.

While I am discussing the use of the current concurrency model (cancel all pending workflow runs except the latest), until and unless that gets the green light our workflow runs will continue to risk colliding and corrupting the shared deployment environment with competing service updates.

[gnuman]

Thanks for the feature request, we need this feature

[meza]

Another usecase for this would be to process a larger amount of pull requests (like from dependabot or renovate) in a sequential manner.

Test/Verify PR -> Merge to Main -> update all PRs -> repeat (with one of the outstanding PRs only without kicking off all the jobs in parallel and then cancelling all but the last)

[shinebayar-g]

I would've stay on GitLab if it's possible. Actions are miles behind what Gitlab CI was 5 years ago lol.

[Denny-g6labs]

+1

[adamatsolita]

+1 "cancel-pending:false"

[SaQQ]

+1 "cancel-pending:false"

[kgomez-bitso]

+1 for ability to set cancel-in-queue: false

[javijvj]

+1 for ability to set cancel-in-queue: false

[vt89]

Unbelievable how this basic feature is not available and how no one from GitHub is replying here.

[MostefaKamalLala]

We are all waiting.

[cmyllis]

2 years later, no fix / workaround ?

[nicola-lunghi]

+1 any news on this????