Every conversation about CI cost starts in the wrong place. Somebody notices the runner bill, somebody else suggests self-hosting, and a quarter later the pipeline is the same length and the invoice is slightly smaller.
The invoice was never the cost. For a team of twelve running an eighteen-minute pipeline four times a day, the runner bill is about $1,590 a year and the engineer time is about $165,600. Optimising the first one is a rounding error on the second.
The two columns
The arithmetic is not subtle. A pipeline run is paid for twice: once by the runner, at fractions of a cent per minute, and once by whoever is waiting for it, at something like a dollar a minute fully loaded. The second payment is a hundred times the first, and only the first appears on a bill anyone reads.
This is also why "move to self-hosted runners to save money" is usually a bad trade. It reduces the small column, adds maintenance, and frequently increases the large one, because a self-hosted fleet queues at peak in a way a hosted one does not. If it makes the pipeline faster, do it for that reason and measure that number.
The threshold that changes the shape
The interesting part of the model is not linear. Below some duration, an engineer watches the pipeline: they keep the change in their head, see the failure immediately, and fix it while they still remember what they did. Above it, they start something else.
Ten minutes is the figure most often quoted and it is not a law; it varies by person, by task and by whether there is anything else to pick up. What is not in dispute is that the threshold exists, and that crossing it changes the kind of cost rather than the amount.
Two consequences follow, and the second is the one teams miss.
Small improvements near the threshold are worth more than large ones far from it. Taking a 12-minute pipeline to 9 moves people from switching to waiting. Taking a 40-minute pipeline to 25 does not move anybody: they were going to leave either way. That is why "we halved the pipeline and nobody noticed" is such a common outcome.
A red build survives longer above the threshold. Nobody is watching when it goes red. The next person pushes on top of it, the failure is now attributable to two changes instead of one, and what should have been a one-minute fix is a bisect. That cost does not appear in any duration calculation and it is frequently the largest one.
Work out your own
calculator · CI pipeline cost
What your pipeline costs across a year
annual cost by pipeline duration, same run volume
where the cost sits
The model has one discontinuity in it on purpose. Below the wait threshold, an engineer watches the pipeline and loses exactly its duration. Above it, they start something else and instead pay a context switch to come back — which can make a longer pipeline cost less per run than one just over the line, while being much worse for how the day feels and for how long a change takes to land. Both the threshold and the switch cost are inputs because neither has a defensible universal value; what is not in dispute is that they are not zero. One consequence to read carefully: above the threshold the engineer-time figure stops rising with duration, because the attention has already been lost. That is the model being honest about what it prices, not a claim that a 60-minute pipeline is as good as a 15-minute one — the extra minutes are paid in cycle time, in how long a red build stays red, and in how large a batch each merge becomes, none of which is here. Flaky runs are not modelled and are usually the larger cost: a failure nobody believes costs a full run plus the decision to re-run it.
The flat engineer-time figure above the threshold is the model being honest about what it prices. Once attention is lost it is lost, so the calculator stops charging you for further minutes. That is not a claim that a 60-minute pipeline is as good as a 15-minute one — the extra minutes are paid in cycle time, in how long a red build stays red, and in how much work accumulates behind each merge. Those matter and are not here.
Two experiments:
Set the duration to just under the threshold. The saving is larger than the minutes removed, because the behaviour changes. This is the single most valuable target in most pipelines and it is a much smaller engineering project than "make CI fast".
Set runs per day to 1. The whole problem shrinks, which is the argument people make for batching changes — and it is a bad one, because it trades a measurable CI cost for an unmeasured integration cost that arrives later and larger.
Where the minutes usually are
The order matters because the effort differs by an order of magnitude.
| Change | Typical effort | Typical recovery |
|---|---|---|
| Cache dependencies and build layers | Hours | Often 30–60% of total duration |
| Reorder so cheap checks fail first | Hours | The whole run, on a bad push |
| Split test suites by measured timing | A day | Bounded by your slowest single job |
| Path filters / affected-target detection | Days, and ongoing risk | Large in a monorepo, small elsewhere |
| Fix the flaky tests | Weeks, unglamorous | Not a speedup; usually the biggest win |
Both GitHub Actions and GitLab document the caching and matrix primitives this needs — actions/cache and the matrix strategy on one side, cache:key and parallel:matrix on the other — so this is configuration rather than engineering. Splitting by measured timing rather than by directory is the one people get wrong most often. If you shard a test suite into eight jobs by folder, your pipeline is as long as the slowest folder, and that is usually the one nobody has looked at. Sharding by recorded per-test duration gets you close to total time divided by shards, which is what parallelism was supposed to buy.
quick check
Your pipeline takes 40 minutes. You get it to 20. How much attention do you get back per run, on a 10-minute threshold?
Above the threshold the engineer has already left the task, so the cost is the context switch to return, not the duration. Twenty minutes is still better than forty — for cycle time, for how long a red build stays red, for batch size — but it does not buy back attention. Getting from 12 to 9 does.
What this does not price
Cycle time is a different and often larger cost. How long a change takes to go from written to deployed affects how quickly you learn whether it worked, how big each batch of changes is, and how painful a revert is. A long pipeline inflates all three. That is the DORA argument for lead time, which it defines as "the amount of time it takes for a change to go from committed to version control to deployed in production" — a definition that includes the pipeline and everything after it, and none of which appears in an attention model.
Queueing at peak is invisible until it is not. If every engineer pushes before lunch, the number that matters is not the pipeline duration but the pipeline duration plus the wait for a runner. Self-hosted fleets are especially prone to this, and it is the same queueing arithmetic as any other saturated service — utilisation and concurrency together, not either alone.
Context switch costs are contested. The 15-minute default is a common working figure rather than a settled one; the research on interruption cost — starting with Mark, Gudith and Klocke's study of interrupted work, which found interrupted tasks were completed faster but at the cost of more stress and effort — measures something adjacent to what happens when a pipeline finishes without you. Treat it as a parameter to argue about, not a constant.
Not all waiting is lost. Some engineers use a ten-minute pipeline to review someone else's pull request, and that is genuinely productive. The model charges for the time regardless, which overstates the cost for teams with a healthy review queue and understates it for teams whose engineers just sit there.
The hourly rate is contentious and it is the biggest multiplier here. Sixty dollars an hour fully loaded is a defensible mid-point in many markets and wrong by a factor of three in others. Put your own in; the ratio between the two columns barely moves, which is the finding that survives.
Making the pipeline faster is not always the right project. If your pipeline is eight minutes and your code review takes two days, the pipeline is not your delivery problem. Measure where the time in a change's life actually goes before optimising the part that has a dashboard.
questions people ask
How much does a slow CI pipeline actually cost?
For twelve engineers running an 18-minute pipeline four times a day — 11,040 runs a year — about 2,760 engineer-hours, or roughly 1.7 full-time people, against about $1,590 of runner compute. The engineer time is the larger number by around a hundred times.
Is self-hosting CI runners cheaper?
It reduces the smaller of the two costs and often increases the larger one, because a fixed fleet queues at peak. Do it if it makes the pipeline faster or unblocks something you cannot do on hosted runners, and measure the duration rather than the invoice.
What is a good CI pipeline duration?
Under the threshold at which people stop waiting, which is around ten minutes for most teams. The exact figure matters less than which side of it you are on, because crossing it changes the behaviour rather than just the number.
What is the fastest way to speed up a pipeline?
Caching dependencies and build layers, then reordering so that cheap checks fail before expensive ones. Both are a few hours of work and together often remove more time than anything architectural. Splitting test suites by measured timing rather than by folder is the next one.
Do flaky tests matter more than pipeline speed?
Usually. A 2% flake rate on 11,000 runs is 220 false failures a year, and the real damage is that people learn to re-run before reading, which makes every genuine failure slower to diagnose. That habit is not fixed by a faster pipeline.
Should we batch changes to run CI less often?
It reduces a cost you can measure and increases one you cannot. Larger batches mean harder reverts, more coupled failures and slower feedback on whether a change worked. Almost always the wrong trade.
The short version
The runner bill is about one per cent of what a slow pipeline costs. The other ninety-nine per cent is engineer time, and for a team of twelve with an eighteen-minute pipeline it is roughly two full-time people a year.
That time is not lost linearly. There is a threshold — around ten minutes — below which people watch the pipeline and above which they leave and pay a context switch to return. So the highest-value optimisation is usually not "make CI fast" but "get under the threshold", and improvements that stay above it buy much less than their minutes suggest.
Then fix the flakes. It is not a speedup and it is usually worth more than one, because a pipeline nobody believes is a pipeline nobody reads.