Replies: 71 comments 32 replies
-
|
Hi @Shereef - thanks much for the feedback. Our initial design was focused on deployment scenarios, but we want to address the use case that you've outlined. I appreciate you outlining this so nicely and providing these suggestions. I hope that this is an area that we can address in the future. Thanks again. |
Beta Was this translation helpful? Give feedback.
-
|
Any updates on this? |
Beta Was this translation helpful? Give feedback.
-
|
Any updates on this? |
Beta Was this translation helpful? Give feedback.
-
|
@chrispat @tuves |
Beta Was this translation helpful? Give feedback.
-
|
@chrispat @tuves Once again asking if anyone is following up on this feature since ethomson left Github? |
Beta Was this translation helpful? Give feedback.
-
|
@ankneis could you see if anyone has anyone taken this on since ethomson left Github please? |
Beta Was this translation helpful? Give feedback.
-
|
The desired behavior here, I believe, would be for jobs with the same |
Beta Was this translation helpful? Give feedback.
-
|
The OP is asking for jobs/workflows to wait and execute sequentially instead of canceling. Canceling existing workflows when a new workflow is triggered is already supported: https://docs.github.com/en/enterprise-server@3.3/actions/using-jobs/using-concurrency
If your issue is specific to manually rerunning the same workflow, seems like a separate issue. |
Beta Was this translation helpful? Give feedback.
-
|
The said behavior is problematic as it's canceling it's own jobs in the same run. I have 6 jobs to run in each commit and they cancel each other in the same run. |
Beta Was this translation helpful? Give feedback.
-
|
Can your issue be solved by using concurrency on the workflow level instead of the job level? Afaik concurrency on the job level is meant more for branching paths (like their fallback value example). Sorry if I'm misunderstanding your usecase though |
Beta Was this translation helpful? Give feedback.
-
|
|
Beta Was this translation helpful? Give feedback.
-
|
I think I just need the same, but add to a queue instead of cancel the workflow. I meant, don't run un parallel but in serial instead This will help, for example, use the same build result or cache in multiple runs for the same commit. The first one will create the artifacts/cache and the next one will be faster |
Beta Was this translation helpful? Give feedback.
-
|
That will complicate things even further. |
Beta Was this translation helpful? Give feedback.
-
|
Cache management is orthogonal to concurrency. Cache management across multiple jobs is something the workflow writer has to think about. Controlled concurrency among jobs in the same concurrency group, which is what this feature request is about, is not possible with what GHA provides out of the box. |
Beta Was this translation helpful? Give feedback.
-
|
Was surprised to see that canceling previous runs was the only option officially supported -- waiting and executing sequentially is what I need and seems to me like a clear usecase (canceling a deployment could result in a broken state) So far I've found this Action: https://github.com/ahmadnassri/action-workflow-queue But ideally, workflows sitting waiting for previous workflows to finish wouldn't consume action minutes in an official feature. |
Beta Was this translation helpful? Give feedback.
-
|
What I needed was exactly what https://github.com/styfle/cancel-workflow-action provided |
Beta Was this translation helpful? Give feedback.
-
|
We implemented something similar with a It's disappointing that the choice is between scaling up our self-hosted runners or surpassing our build minutes quota for github-hosted runners or building a separate queue outside of github actions to handle this kind of functionality. |
Beta Was this translation helpful? Give feedback.
-
|
We have a long running workflow that goes through several jobs to deploy AWS infrastructure. If one of them was cancelled mid-way, or deploys happened on to of one another, this would cause some pretty serious issues. We definitely need a way to queue jobs in sequence without consuming action minutes |
Beta Was this translation helpful? Give feedback.
-
|
Brainstorming - Eventually I will probably have to implement a specific self-hosted group. Maybe like 25 runners on a t3.small. Their job would be to handle queues. So I'd have a workflow or job dedicated to queing, and my larger runners (c7a.xlarge) wouldn't have to sit and wait for 15 mins at a time - instead I can have a single t3.small for all repos. |
Beta Was this translation helpful? Give feedback.
-
|
@justin-pierce the action-workflow-queue has an issue when there are multiple workflows started in the same second. See #45. @koreyGambill exactly, but do you know about some simple yet reliable queuing solution for that? |
Beta Was this translation helpful? Give feedback.
-
|
Would be great to be able to queue workflow runs (ideally without consuming action minutes). Was surprised that new runs cancel existing ones by default. |
Beta Was this translation helpful? Give feedback.
-
|
We hit this limitation while testing out the Merge Queue feature that is currently in limited beta. Our use case is ensuring that only one workflow is running for the PR at the top of the merge queue. When concurrency is set at the workflow level (for the merge_group event type) and multiple PRs are queued within the merge queue, the system cancels queued check suites once a new PR is added to the queue. It would be fantastic if we had the ability to disable canceling queued jobs! |
Beta Was this translation helpful? Give feedback.
-
|
If you're using self-hosted runners, a workaround/hack we used was to label one of these runners specifically, and then make all the jobs that need to be in a concurrency group just run on this labelled runner instead. No concurrency shenanigans. That causes them to queue up globally across all workflows without any caps on number of waiting jobs. The only major limitation is that if this runner fails, these jobs won't use any other runner. That was considered an acceptable risk to unblock us until we had a better solution. |
Beta Was this translation helpful? Give feedback.
-
|
Well, this is possible only for cases, where you need to serialize only a single job. |
Beta Was this translation helpful? Give feedback.
-
|
My team also needs this feature. |
Beta Was this translation helpful? Give feedback.
-
|
My team also needs this. |
Beta Was this translation helpful? Give feedback.
-
|
My team also need this feature - mainly for queueing mulitple actions each with Terraform plan/deploy steps which cannot be run at same time due to a lock on terraform state. |
Beta Was this translation helpful? Give feedback.
-
|
A must have for my team |
Beta Was this translation helpful? Give feedback.
-
|
Really hard to run terrafrom/terragrunt at scale without it |
Beta Was this translation helpful? Give feedback.
-
|
Really needing this to be implemented. I am currently using https://github.com/ben-z/gh-action-mutex for now, but this does pollute the git history. |
Beta Was this translation helpful? Give feedback.
-
|
Need this feature too. |
Beta Was this translation helpful? Give feedback.
-
|
GitHub, I'll add my team's need for this feature as well. I see a LONG list of replies like this, with no response from GitHub. (I'm in DevOps.) I'm going to put in a GitHub support ticket referencing this request to see if there are any plans to implement it. Our use cases is that I have created a highly parallelized building and deploying four distinct services in a monorepo. In this workflow, the update_service action is called by the jobs for each of those services, AND we have a very active CI/CD environment which can be processing several merges almost simultaneously. While I am discussing the use of the current concurrency model (cancel all pending workflow runs except the latest), until and unless that gets the green light our workflow runs will continue to risk colliding and corrupting the shared deployment environment with competing service updates. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for the feature request, we need this feature |
Beta Was this translation helpful? Give feedback.
-
|
Any news???
the solutions using concurrency would be: concurrency: |
Beta Was this translation helpful? Give feedback.
-
It doesnt exactly works that way. You can trigger 2 jobs, but when you trigger a third, the second will be cancelled. If you have it as true, the first will be cancelled when you trigger a second. |
Beta Was this translation helpful? Give feedback.
-
|
This is not what I am seeing. The in progress job is always cancelled even if I set ""cancel-in-progress: false". |
Beta Was this translation helpful? Give feedback.
-
|
@pchico83 This is how it is set in mine. `concurrency: run-name: Running Test |
Beta Was this translation helpful? Give feedback.
This comment was marked as off-topic.
This comment was marked as off-topic.
This comment was marked as off-topic.
This comment was marked as off-topic.
This comment was marked as off-topic.
This comment was marked as off-topic.
This comment was marked as off-topic.
This comment was marked as off-topic.
This comment was marked as off-topic.
This comment was marked as off-topic.
-
|
I voted on the issue, but the flood of plus one comments seems pretty impressive too. So, "hey, I want multiple overlapping jobs to queue and run serially instead of terminating" as well. I'm pretty sure this is possible, because one job will queue and multiple jobs waiting on runners will queue. So all the necessary logic seems like it's essentially present somewhere. 😅 |
Beta Was this translation helpful? Give feedback.
-
|
The more I try use GitHub actions for advanced use cases the more I stumble onto community conversations like this.
Yet, years later and still... nothing... Not so much as a reply with whether the feature is planned, on the roadmap at all or if this is something we should just learn to live with. |
Beta Was this translation helpful? Give feedback.
-
|
Actually, in this case, the silence is a very clear answer. It looks to me like Actions isn't the first class citizen, and it's not planned to change the state any soon. |
Beta Was this translation helpful? Give feedback.
-
|
+1 |
Beta Was this translation helpful? Give feedback.
-
|
Stop it. This thread don't need the spam that |
Beta Was this translation helpful? Give feedback.
-
|
@jeliasson is correct, please remember to use the upvote button or an emoji instead of creating a new thread to agree with a comment. It keeps the discussion clean. Thank You community!!! |
Beta Was this translation helpful? Give feedback.
-
|
Hey, sorry the Actions team has been quiet on this. We have added this to the list of paper cuts we are hoping to address in the next 12 months :) I don't have a timeframe for this one, but I want to say we know it's a pain point and do have it on our list <3 |
Beta Was this translation helpful? Give feedback.
-
|
Sadly that reply is meaningless vapor without any action whatsoever. |
Beta Was this translation helpful? Give feedback.
-
|
Good to see a reply, @nebuk89 . This is really a LOT more than a paper cut; it's a fact that many workflows are incrementally changing/updating things like databases, and MUST run in sequence. The current design presumes that later workflows are just an addition to the same codebase, and so are expected to produce the same result whether or not an intervening run is cancelled. |
Beta Was this translation helpful? Give feedback.
-
|
Thank you for keeping an eye and replying. Sorry I don't have solid updates yet. So thank you for taking the time <3 please keep engaging and never stop being passionate about us making Actions the best product it can be. |
Beta Was this translation helpful? Give feedback.
-
|
💬 Your Product Feedback Has Been Submitted 🎉 Thank you for taking the time to share your insights with us! Your feedback is invaluable as we build a better GitHub experience for all our users. Here's what you can expect moving forward ⏩
Where to look to see what's shipping 👀
What you can do in the meantime 💻
As a member of the GitHub community, your participation is essential. While we can't promise that every suggestion will be implemented, we want to emphasize that your feedback is instrumental in guiding our decisions and priorities. Thank you once again for your contribution to making GitHub even better! We're grateful for your ongoing support and collaboration in shaping the future of our platform. ⭐ |
Beta Was this translation helpful? Give feedback.
-
|
Hi all! I have a more complex desired behavior: We have a step that deploys with terraform. We want that if a running job already reached that step, the other ones should wait instead of killing it. Ex:
Deployment succeeded, version |
Beta Was this translation helpful? Give feedback.
-
|
@hschimpf what about merge queues ? For a TF deploy it seems better to me to run the full deploy for each commits |
Beta Was this translation helpful? Give feedback.
-
|
Per original request: my team needs something like
Motivating ExampleI have a 'deploy' GitHub Workflow that takes 600 seconds to run a deployment.
Requirements
Expected Behavior
Documented Behavior
|
Beta Was this translation helpful? Give feedback.
-
|
I can understand why it is so difficult to implement. There are many bad jobs out there, which run when they should not. So current behavior prevents them from overwhelming Github servers. Especially if you have heavy jobs running after every commit. But at the same time there are people out there who are willing to throw money at you just for using your CPU. So this is clearly revenue possibility for you. So why it was not implemented already for paid clients, is funny mystery. For people who will cry , that this unfair - do you imagine Github allowing to queue jobs faster than they can execute for free? |
Beta Was this translation helpful? Give feedback.
-
|
That can be easily addressed. You can have explicitly defined queues and set limits on queue sizes and you can add rate limits. This would discard excessive runs of badly behaving jobs. You can implement something like this with what's already in place, but it's tedious and more resource consuming than if Github would do it. So it can't be that difficult. Only, it won't appear out of nothing. Somebody needs to implement it, and apparently MS isn't willing to pay for it. |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
Current 8000 behavior
Desired behavior
Sorry if this was discussed before I tried to search but didn't find anything.
Our current implementation
To give it more context:
Imagine this scenario:
We have 5 PRs for Team A
We merge to main and update the 5 PRs
All 5 will want to deploy to the env + the main branch deploy that is ongoing
the last PR to merge is going to be queued
all other PRs will be cancelled and have to be retried manually
Suggested solutions
one of the below would work
cancel-in-queue: falsea new command to disable cancelling queuing jobscancel-in-progress: falsewould not cancel anything and leave everything in queueBeta Was this translation helpful? Give feedback.
All reactions