ai engineering

Does routing to a cheaper model actually save money?

Trying a cheap model first saves 60% at a 20% escalation rate, and nothing at 80%. That threshold is one minus the price ratio, and it decides the whole design.

Oct 2026

What a million embeddings actually costs in Postgres

Measured on pgvector 0.8.6: a million 1536-dimension vectors is 785 MB per 50,000 rows once the index and TOAST are counted. The estimate misses two thirds.

Oct 2026

What invalid output costs once you retry it

Retrying schema failures costs less than you think and hides more than you expect. What strict mode guarantees, what it quietly does not, and what to validate.

Sep 2026

What batching is worth on your volume

Every provider prices batch at half. The discount is not the variable — how much of your work tolerates a 24-hour ceiling is, and whether it covers the queue.

Sep 2026

What your tools parameter costs per call

Tool definitions ride on every request and are never deduplicated. What 58 tools cost per call, and why caching settles the bill but not the context window.

Sep 2026

Do you need an LLM gateway?

What an LLM gateway gives you, what it costs in fees and in latency, and a checklist for choosing between a library, a self-hosted proxy and a hosted one.

Sep 2026

What one agent task costs, step by step

An agent resends its history on every step, so cost grows with the square of the step count. What a task costs, and where to put the caps.

Sep 2026

How many test cases does your eval need?

Your eval went from 80% to 85% on 40 cases. That is noise. The sample size maths for LLM evals, with a calculator and the intervals nobody prints.

Sep 2026

Retrieval or the whole corpus in the prompt?

Retrieval, fine-tuning and a million-token context window solve different problems. A cost comparison, the break-even corpus size, and when each one fails.

Sep 2026

What prompt caching saves on your traffic

How prompt caching works on all three big providers: minimum prefixes, lifetimes, write premiums, prompt ordering, and the hit rate where caching starts paying.

Sep 2026

Scope your chatbot, see what it costs

Agency guides quote anything from $5,000 to $300,000. Here is where the money actually goes, line by line, with an estimator and the monthly bill after launch.

Sep 2026

What your AI feature costs per user

Tokens per call, calls per action, actions per user: a calculator for what an AI feature costs each user, and why the heaviest 10% decide whether the plan works.

Sep 2026

What tracing your LLM calls will store, and what each call costs

Trace model calls with OpenTelemetry's GenAI conventions, attribute cost per user at 2026 prices, and size trace storage before prompt payloads become most of it.

Mar 2026

architecture

Will your breaker trip on noise, or miss the outage?

A 10-request window at a 50% threshold, against a dependency that errors 5% of the time, opens on pure noise 110 times a day. Twenty requests: once in 100 days.

Oct 2026

What moving your data actually costs

Twenty terabytes to the internet costs $1,783 a month. The cross-AZ traffic and NAT gateways behind it cost $1,321 more, and nobody budgeted for either.

Oct 2026

What your pipeline costs across a year

Twelve engineers and an 18-minute pipeline lose 2,760 engineer-hours a year, against $1,590 of runner time. The compute is the smaller number by 104 times.

Oct 2026

Is your rotation big enough, or just quiet enough?

Twelve pages a week at six hours each is 72 engineer-hours. Across five people that is 36% of everyone's week, against a published ceiling of 25%.

Oct 2026

Does your timeout catch failures or create them?

A two-second deadline across four hops with one retry each leaves 245 ms per attempt. The P99 of one hop is 300 ms. That timeout is not catching failures.

Oct 2026

Order a strangler fig migration

How to move off a legacy monolith without a rewrite: the facade, what order to move modules in, a wave planner, and the reasons migrations stall.

Sep 2026

Build it or buy it, over three years

Auth, billing, search or notifications: what building really costs over three years once maintenance is counted, and the test that comes before the money.

Sep 2026

Freelancer, agency, fractional CTO or a hire

Four ways to build or fix a product, compared on published rates: what each delivers in six months, what each costs an hour, and how each one fails.

Sep 2026

What an hour of downtime costs you

Surveys put an hour of downtime above $300,000. Here is the arithmetic for a startup instead: interrupted revenue, SLA credits, people, support and churn.

Sep 2026

Score your company before a reviewer does

What investors and acquirers check in your tech: architecture, security, data, delivery, bus factor and cost. A 20-question scorecard and what to fix first.

Sep 2026

How many customers are you charging twice?

A timeout does not mean the request failed. How idempotency keys stop retries creating duplicate charges and orders, what to store, and a calculator for the damage.

Mar 2026

Are you ready for microservices?

Every service on a request path multiplies failure and slow tails. A readiness scorecard, the distributed tax in numbers, and when a modular monolith is the answer.

Mar 2026

system design

When should you start refusing work?

At 90% utilisation one worker turns a 40 ms request into 400 ms. Forty workers turn it into 44 ms. The advice everyone repeats depends on a number nobody states.

Oct 2026

Does this index pay for itself?

Eight indexes cut insert throughput by 70% in my measurements, and one on the wrong column took heap-only updates from 89,555 to zero. Here is the ledger.

Oct 2026

What hits your database the moment the cache expires

Forty requests a second on one key with a 1.5-second recomputation is sixty simultaneous queries the moment it expires. Jitter fixes the wrong half of that.

Oct 2026

How many deploys does your SLO pay for?

Three nines is 43 minutes a month. A bad deploy that takes 45 minutes to mitigate at 30% impact spends 14 of them. The rest is arithmetic nobody does.

Sep 2026

What your requests reserve, and what your limit throttles

Requests decide what you pay for. Limits decide when the kernel stops you. Most teams set both to the same made-up number and pay twice for it.

Sep 2026

What a rate limiter does to a burst

A token bucket forgives a burst and still holds the average. Fixed windows let a client send double at the boundary. A simulator, and the headers to send back.

Sep 2026

Will your pools fit in max_connections?

Eight app instances with pools of ten is eighty connections to a database that can usefully run eight. The sizing formula, max_connections and pgbouncer.

Sep 2026

What fan-out does to your tail latency

A backend that is slow 1% of the time makes 63% of user requests slow when one request waits on a hundred of them. The fan-out maths, and how hedging fixes it.

Sep 2026

How many workers does this queue need?

One formula sizes worker pools, connection pools and queues: L = λW. What it does and does not tell you, why the last worker matters most, and a calculator.

Sep 2026

What happens when everyone retries at once

Retries without jitter arrive in waves and stretch a ten-second outage into four and a half minutes. The four backoff formulas, a simulator, and what to set.

Sep 2026

What can you actually promise?

Every service a request touches multiplies into your uptime. A composite SLA calculator, the published numbers for AWS, GCP, Azure and Twilio, and what to promise.

Sep 2026

How long until your queue runs out of memory?

A queue with no ceiling grows at the difference between two rates until the process dies. The arithmetic, a simulator, and the four places the extra work can wait.

Mar 2026

Will this backlog ever drain?

A queue drains only by the capacity above its incoming rate. Work out drain time, message wait and consumers needed, and where Kafka's partitions stop you.

Mar 2026

Need the number for your own system?

The calculators cover the common shapes. If yours is stranger than that, tell me what you are building and I'll work it through with you on a call.