Timeouts get set the way office thermostats get set: somebody picks a number that feels reasonable, and then everyone lives inside the consequences without connecting them back to the number. Thirty seconds, because that was the default. Five seconds, because thirty felt too long. One second, because someone read that fast failure is good.
The number that matters is not any individual timeout. It is whether all of them, multiplied by all the retries, fit inside what the caller is actually willing to wait — and whether what is left over is still longer than a healthy request takes on a bad day. Those two conditions pull in opposite directions, and most chains satisfy neither.
The two divisions
Start from the only number with a real owner: how long the caller will wait. For a page load that is a product decision, somewhere around two seconds. For a mobile app doing a background sync it might be thirty. Everything downstream is spending that budget.
The first division is by depth. If the request passes through four services, each one gets a quarter of the budget — assuming nobody owns the whole thing, which is the usual case. The second division is the one teams forget: if a hop retries, its slice has to fit every attempt, not one.
Two seconds, four hops, one retry each: 2000 ÷ 4 ÷ 2 = 250 ms, minus network overhead, so about 245 ms per attempt. If a single hop's P99 is 300 ms — a completely ordinary figure for a service that talks to a database — then the slowest 1% of perfectly healthy calls now time out.
What happens next is worse than the timeout. The caller retries, the retry is also a slow request because the slowness was real, and it times out too. The request fails after consuming two full attempts' worth of work at every level. The service is healthy, the database is healthy, and the error rate is not zero.
A timeout below the P99 is a failure generator
This is the single most useful thing to check, and almost nobody checks it: is your timeout longer than the latency you consider acceptable?
If a request that takes 280 ms is fine — nobody complains, no SLO is violated, the data is correct — then a 245 ms timeout is a policy that 1% of your successful work should be thrown away and done again. The Google SRE book puts the tension plainly: "Short deadlines can cause some more expensive requests to fail consistently. Balancing these constraints to pick a good deadline can be something of an art." It is equally direct about the other end: "Having deadlines several orders of magnitude longer than the mean request latency is usually bad."
So the working range is narrow. Above the P99 of legitimate work, and well below the point where a hung request occupies a thread for a minute.
| Timeout relative to P99 | What it does | Where it belongs |
|---|---|---|
| Below P99 | Fails healthy slow requests, then retries them | Nowhere |
| 1.0 to 1.5 × P99 | Catches genuine hangs, occasional false positive | Tight budgets, idempotent calls |
| 2 to 3 × P99 | Comfortable; still bounded | Most internal RPCs |
| 10 × P99 or more | Threads pile up before anything fires | Only where the work is genuinely variable |
The awkward implication: if the budget does not leave room for a timeout above your P99, the answer is not a cleverer timeout. It is one of fewer hops, a faster hop, fewer retries, or a longer deadline. The calculator will tell you which of those you need, and it is usually "fewer retries".
Retries multiply down the chain
The other half of the problem is what those retries do to the services underneath you.
A thousand requests a second at the edge becomes sixteen thousand at the bottom, in the worst case where everything fails late. That is the mechanism behind a very common outage shape: the database gets slow, every layer above it starts retrying, and the load the database is struggling with goes up by an order of magnitude. It does not recover on its own, because the recovery would require the load to drop, and the load is now made of retries.
The fix is not "never retry" — it is to retry in one place. Retrying only at the edge keeps the multiplier at 2× for the entire chain instead of 16×. Where a deep retry really is worth it, bound it with a retry budget and jittered backoff so that the multiplier is a rate, not an exponent.
Deadlines, not timeouts
There is a structural fix that removes a whole class of this, and it is one line in most RPC frameworks.
A timeout is a duration: "give up after 500 ms". A deadline is a point in time: "give up at 14:03:07.412". If each hop sends a fresh timeout, every hop restarts the clock, and a request can keep doing work long after the client stopped waiting. If each hop passes the remaining time down, the whole tree shares one expiry.
The SRE book's example is exact: "If server A selects a 30-second deadline, and processes the request for 7 seconds before sending an RPC to server B, the RPC from A to B will have a 23-second deadline." And on the cost of skipping it: server C "processes the request thinking it has 15 seconds to spare, but is not doing useful work."
gRPC makes this the default model — "A deadline is used to specify a point in time past which a client is unwilling to wait for a response from a server" — and it handles the arithmetic for you: "gRPC converts the deadline to a timeout from which the already elapsed time is already deducted. This shields your system from any clock skew issues." It also warns about the other default: "By default, gRPC does not set a deadline which means it is possible for a client to end up waiting for a response effectively forever."
Over HTTP there is no standard header for this, so teams pass their own — a X-Request-Deadline with an absolute timestamp, or a remaining-milliseconds value that each hop decrements. It is unglamorous and it removes an entire category of wasted work.
The second half is checking it. The SRE book again: "If handling a request is performed over multiple stages... the server should check the deadline left at each stage before attempting to perform any more work on the request." A deadline that is propagated but never inspected saves nothing.
Work out your own chain
calculator · timeout budget
Does your timeout catch failures or create them?
deadline remaining at each hop when every hop runs at its P99
worst-case requests reaching each hop, per request the client sent
The equal split across hops is deliberately naive: it is what teams do when nobody owns the whole budget. A real chain is lopsided, and the right allocation gives the slow hop more and the cache lookup less — the calculator's job is to show whether any allocation fits at all. Amplification is the worst case, where every attempt fails late rather than fast; a circuit breaker or a retry token bucket cuts it well below this, which is exactly why those exist.
Three things to try:
Set retries to 0. The per-attempt timeout doubles to 495 ms, comfortably above the 300 ms P99, and the amplification drops to 1×. This is why "retry at the edge only" is such a common recommendation from people who have operated large systems: it buys back the entire budget for everyone below.
Add a hop. Five services instead of four, and the same budget gives 195 ms per attempt. This is the hidden cost of splitting services too early: every boundary you add spends latency budget that has to come from somewhere, and it usually comes out of the timeout headroom nobody is watching.
Raise the P99 to 500 ms. Nothing fits any more. That is the honest signal that your problem is not timeout configuration — it is a hop that is too slow for the deadline you have promised, and no arrangement of timeouts will fix it.
Allocating the budget properly
The equal split is a simplification, and a real chain should not use it. A cache lookup with a 5 ms P99 does not need a quarter of the budget, and the service that talks to a payment provider needs more than its share.
A better allocation, in order:
- Start from the user-facing deadline and subtract what the edge itself spends: TLS, auth, serialisation, the response render. What remains is what the chain can spend.
- Give each hop a timeout of about 2× its own measured P99, not an equal slice. Add them up.
- If the sum exceeds the budget, you have a design problem, not a configuration problem. Fix it by removing a hop, parallelising two of them, caching one, or renegotiating the deadline.
- Add retries only where the sum leaves room, and preferably at one level only.
- Propagate the deadline so that the hops which run late shorten everything below them automatically.
- Set the timeout at the caller, not the callee. Only the caller knows how long it is willing to wait; the callee only knows how long it usually takes.
Step 3 is the one people skip, and it is the only step that finds real problems. An arrangement of timeouts that does not add up is not saved by tuning the individual numbers.
There is one more subtlety worth knowing if you run a proxy in front of all this. Envoy distinguishes the route timeout — "the amount of time that Envoy will wait for the upstream to respond with a complete response", default 15 seconds — from per_try_timeout, which bounds each attempt when retries are configured. If you set only the overall timeout, retries share it; if you set only the per-try timeout, the total is unbounded. Both, or you have not bounded what you think you have.
quick check
Your P99 for one hop is 300 ms and your per-attempt timeout is 245 ms. What does the timeout do during a normal, healthy day?
The P99 is the level 1% of requests exceed. A timeout below it fires on those requests even when everything is healthy, so 1% of successful work is thrown away and retried — doubling the load those requests create while turning some of them into errors. This is the failure mode that looks like a flaky dependency and is actually a configuration choice.
What this arithmetic ignores
Hops in parallel do not add. If your service fans out to three backends at once, the chain cost is the slowest of the three, not the sum. The calculator assumes a sequential chain because that is where budgets usually go wrong, but a fan-out has its own and nastier problem: the response waits for the slowest of N, so the p99 of the whole is much worse than the p99 of each. That is the tail latency of a fan-out, and it makes the budget question harder rather than easier.
Queueing time is not in these numbers. A hop's P99 measured at low load is not its P99 when the retries arrive. As utilisation rises, waiting time rises faster than service time, so the very moment your timeouts start firing is the moment the P99 they were sized against stops being true. Sizing against a quiet-hour P99 gives a budget that dissolves under load.
Connection establishment is not free. A cold TLS handshake can cost more than the request. The overhead field covers a warm connection; if your pool is thrashing, the real per-attempt cost is much higher and the budget is tighter than the calculator suggests.
Retries are only safe where the operation is idempotent. All of the above assumes retrying is the right thing to do. Retrying a payment charge because the response was slow is a different kind of incident, and the timeout arithmetic is the least of the problems there.
questions people ask
How do I calculate a timeout for a service in a call chain?
Start from the caller's deadline, subtract what the edge spends on itself, then allocate what remains across hops in proportion to their measured P99 rather than equally. Give each hop roughly twice its own P99, and if the total exceeds the budget you have a design problem rather than a configuration one. Then check that retries fit: a hop's slice must contain every attempt, not one.
Should my timeout be based on P99 or P50 latency?
P99, with headroom. A timeout below the P99 fires on healthy requests that were merely slow, converting successful work into errors and retries. Two to three times P99 is a comfortable range for most internal calls.
What is deadline propagation and do I need it?
It means passing the remaining time down the chain rather than a fresh timeout at each hop, so every service shares one absolute expiry. Without it, a service can keep working long after the client has given up. gRPC does it natively and deducts elapsed time automatically; over HTTP you pass your own header.
How much load do retries actually add?
In the worst case, attempts per hop raised to the number of hops. One retry across four hops is 16× at the deepest service. That worst case only occurs during an incident, which is precisely when the deepest service has the least capacity to absorb it.
Should I retry at every layer or just one?
One, as close to the user as you can manage. Retrying at every layer multiplies; retrying at the edge does not. Where a deep retry is genuinely valuable, bound it with a retry budget so it becomes a rate rather than an exponent.
What timeout should I set if I do not know my P99 yet?
Measure before you choose. If you must ship something first, set it generously — several times the mean rather than a fraction of it — and add an alert on requests that exceed it, because a too-long timeout is visible and recoverable while a too-short one looks like an unrelated flaky dependency.
The short version
A client deadline is divided twice: by hops, and by attempts. Two seconds over four hops with one retry each leaves about 245 ms per attempt, and if one hop's P99 is 300 ms then that timeout fires on healthy requests, retries them, and fails them again.
Check one thing first: is every timeout longer than the latency you would be happy with? If not, the timeout is not protecting you from anything — it is generating the errors you are seeing.
Then check what your retries multiply out to. One retry per hop across four hops is 16× at the bottom, arriving exactly when the bottom is already struggling. Retry in one place, propagate the deadline rather than resending a duration, and make each hop check the remaining time before it starts work it cannot finish.