Short answer: AI cost is a design choice, not a market condition
Spend follows your decisions. Prices for a fixed level of capability fall, but a bill depends on the model tier you choose and the tokens each task uses. Four controls and a shared data foundation hold a two-year cost envelope. In our model, inference is then only about 11 per cent of programme cost.
Last reviewed: 27 October 2026. Modelled analysis, not a forecast.
- Tier matters most. The same task costs 114 times more at the frontier tier than at the budget tier.
- Self-hosting rarely wins. It needs sustained utilisation above about 37 per cent and enormous volume.
- Controls compound. The programme with routing, caching and capacity planning costs about 2.8 times less than the one without.
- Staff cost dominates. Engineering is the largest line in every scenario, so reuse beats repetition.
Why do AI bills rise while model prices fall?
Spend is tasks multiplied by tokens per task multiplied by price per token, and only the last factor is falling. Stanford HAI recorded a fall of more than 280 times in the price of fixed capability between November 2022 and October 2024, while reasoning, retrieval and agents multiplied tokens per task.[1]
Capability at a fixed level gets cheaper, but each model generation adds a tier above the last. In July 2026 the most capable models cost roughly $10 per million input tokens and $50 per million output tokens, against $5 and $25 a year earlier.[8]
Agent workflows resubmit context at every step, and Gartner estimates agentic workloads use five to thirty times the tokens of a single-turn interaction.[2] A fivefold price fall against a sevenfold rise in tokens still leaves a task costing 40 per cent more. Tier is the bigger lever: small to frontier multiplies cost by ten.
How much does the model tier matter?
More than any price change. Published pricing spans more than a hundredfold between the cheapest and most expensive tiers. For a structured extraction at 10 million documents a year, the cost runs from about £2,630 at the budget tier to about £300,750 at the frontier, a factor of 114 for identical work.[8]
| Tier | Indicative rate per million tokens | Cost per document | Annual cost |
|---|---|---|---|
| Budget | $0.10 input, $0.40 output | $0.00035 | £2,632 |
| Small | $1.00 input, $5.00 output | $0.00400 | £30,075 |
| Mid | $3.00 input, $15.00 output | $0.01200 | £90,226 |
| High | $5.00 input, $25.00 output | $0.02000 | £150,376 |
| Frontier | $10.00 input, $50.00 output | $0.04000 | £300,752 |
Data Hatch model. Assumes 1,500 input and 500 output tokens per document, indicative July 2026 list rates and $1.33 to £1. Rates vary by provider.[8]
The buyer decides which row applies. The costliest default is a pilot built on the frontier tier that moves into production unchanged. Route to the smallest sufficient tier and escalate by exception.
When does self-hosting beat buying tokens?
Only at high sustained utilisation and very large volume. At 80 per cent utilisation, dedicated infrastructure beats a small purchased tier past about 530 billion tokens a year. Below about 37 per cent utilisation it never wins at any scale, because the hardware alone costs more than the tier.
- Engineers, not hardware, set the cost. At one accelerator unit, engineering is about 90 per cent of the all-in $8.83 per million tokens at 80 per cent utilisation. At 32 units the figure is $1.58.
- Utilisation scales cost directly. Halve it and the rate per token doubles. Interactive workloads typically reach only 20 to 40 per cent.
- The floor is hardware alone. Hardware-only cost is $0.74 per million tokens divided by utilisation, which passes the $2.00 small-tier rate below 37 per cent.[6][7]
- Beating the frontier proves little. Compare with the smallest tier that can do the work.
Ten million documents a year is about 20 billion tokens, twenty-six times below break-even. Buying costs about £30,100 a year, against £225,000 or more for the extra engineers. The saving comes from routing, not hosting, though data residency can still justify self-hosting.
Which four controls hold a cost envelope?
Route work to the smallest sufficient tier, cache repeated context, buy committed capacity sized to measured base load, and negotiate proportionate contract protection. They follow FinOps Foundation practice. Published savings belong to the benchmark that produced them, so measure your own workload rather than assume.[5]
Route to the smallest sufficient tier
RouteLLM cut cost by more than 85 per cent on one benchmark while keeping about 95 per cent of reference performance, but by only 45 and 35 per cent on two others.[3] FrugalGPT reports up to 98 per cent on a single financial-news dataset.[4] Plan against 35 to 85 per cent.
Cache context and batch deferrable work
Providers discount cached reads by about 90 per cent, but writes carry a premium and entries expire, so the blended saving is lower. Work that can wait moves to batch at about half rate.[8]
Commit capacity to base load, not peak
Reserved capacity turns variable billing into a discounted fixed charge. Sized to peak, the discount becomes permanent overpayment on idle capacity.
Negotiate protection
Ask for rate protection for the term, notice of model withdrawal, protection against metering changes, workload portability and periodic benchmarking against market rates.
All four are far cheaper to design in than to retrofit. Give every agent a step limit too: no ceiling on steps means no ceiling on cost.
What does a 24-month cost model show?
Applying the controls cuts the modelled run rate from about £1,385,000 to about £489,000 a year, a ratio of 2.8 times, for identical adoption and outcomes. Inference is then 11 per cent of the total, but 33 per cent without the controls. The model shows the potential size of the effect, not proof.
| Component | Optimised | Routed | Without controls |
|---|---|---|---|
| Inference (documents and interactive) | £26,000 | £56,000 | £451,000 |
| Data platform capacity and storage | £58,000 | £73,000 | £219,000 |
| Observability, evaluation and compliance | £30,000 | £35,000 | £70,000 |
| Platform operation and support | £225,000 | £300,000 | £525,000 |
| Residual integration effort | £15,000 | £25,000 | £120,000 |
| Total annualised run rate | £354,000 | £489,000 | £1,385,000 |
| Inference as a share of total | 7% | 11% | 33% |
Data Hatch model at an illustrative scale, to show relative magnitude and composition. It does not forecast spend for any organisation. Engineers are costed at £150,000 fully loaded.
- Staffing is the biggest line. One extra engineer adds about £150,000 a year, roughly five times the routed document-inference line.
- The gap is conservative. The uncontrolled case assumes the high tier. Defaulting to the frontier takes it to about £1,836,000, or 3.8 times the routed case.
What should you do first?
Measure before you commit. Spend four to six weeks inventorying systems, measuring volumes and token profiles, and costing today's process. Then state the commitment as cost per unit of business work inside a volume band, and build a shared data foundation so each new initiative reuses the last one's integration work.
- Express the commitment per unit. Define the unit (per document, claim or query), the volume band, the review cadence, the exclusions and what triggers renegotiation.
- Instrument from the first workload. Record tier, cached and uncached tokens, agent steps and escalations per request.
- Fix the root cause. Where systems disagree on definitions, every AI project repeats the same integration. A governed definition layer over data that stays in its source systems usually delivers value in three to nine months,
- Give the envelope an owner. Total spend should rise if adoption works, so govern unit cost instead.
Cost predictability is not a clean-up project. It is what you get when the controls and the foundation are built in from day one.
Frequently asked questions
Why do AI bills rise while model prices fall?
Published prices fall for a fixed level of capability, but spend depends on how much computation each task uses and which capability tier it runs on. Reasoning models, retrieval and agent workflows multiply the tokens billed per task, and each new model generation adds a pricier tier above the last. A fivefold fall in price against a sevenfold rise in tokens leaves a task costing 40 per cent more.
How much does the choice of model tier affect AI cost?
More than any price change will. Published pricing spans more than a hundredfold between the cheapest and most expensive tiers. At 10 million documents a year, the same extraction task costs about £2,630 at the budget tier and about £300,750 at the frontier, a factor of 114, and the buyer decides which tier to use.
When does self-hosting AI inference cost less than buying tokens?
Only at high sustained utilisation and very large volume. At 80 per cent utilisation, dedicated infrastructure beats a small purchased tier past about 530 billion tokens a year. Below about 37 per cent utilisation it never wins at any scale, because the hardware alone costs more than the purchased tier charges for the same throughput.
What share of an AI programme's cost is inference?
In the model, inference is about 11 per cent of annual programme cost once routing, caching and capacity planning are applied. Without those controls it rises to about a third of a total that is itself 2.8 times larger. The figures are modelled at an illustrative scale, not forecast for any one organisation.
How do you make AI spend predictable?
Apply four controls from the start: route each request to the smallest sufficient model, cache repeated context, plan capacity against measured base load, and negotiate rate protections. Build workloads on a shared data foundation so integration work is reused, and govern cost per unit of business work inside a volume band rather than total spend.
References
Sources cited in this analysis, listed in the order first relied upon. Pricing and infrastructure references are point-in-time, from July 2026.
- Stanford Institute for Human-Centered AI, AI Index Report 2025, research and development chapter, for the recorded fall in inference cost at fixed benchmark performance between November 2022 and October 2024.
- Gartner, analysis published March 2026, estimating that agentic workloads use 5 to 30 times the tokens of a single-turn interaction. Widely reported at second hand: obtain the primary publication before relying on it contractually.
- Ong, I. et al., “RouteLLM: Learning to Route LLMs with Preference Data”, arXiv preprint 2406.18665, June 2024. arxiv.org
- Chen, L., Zaharia, M. and Zou, J., “FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance”, arXiv preprint 2305.05176, May 2023. arxiv.org
- FinOps Foundation, FinOps Framework, including the FinOps for AI technology category.
- vLLM project, published performance benchmarks for open-weight model serving, the basis for throughput per accelerator (1,500 tokens per second).
- Published list pricing for accelerated compute from specialist cloud infrastructure providers ($3.99 per accelerator hour), retrieved July 2026.
- Published API pricing from major model providers, including tier rates, cached input discounts and batch discounts, retrieved July 2026.
Last reviewed: 27 October 2026.
Modelled analysis. The figures show relative size and behaviour, not a forecast for any specific organisation. Prices change quickly, so confirm current rates before any commitment.