Summary of findings
Model prices are falling quickly, yet enterprise AI budgets keep climbing. Both are true at the same time, and the reason matters more than either fact alone. Falling prices apply to a fixed level of capability. Actual spend depends on how much computation a task consumes and which capability tier it runs on, and both of those are choices rather than market conditions. That makes AI cost far more controllable than the headlines about runaway spend suggest, provided the right controls are in place from the start.
Capability got cheaper. The top tier got more expensive.
Between November 2022 and October 2024, the price of a fixed level of model capability fell more than 280-fold, according to Stanford HAI's AI Index. Over the same period, each new model generation opened a pricing tier above the last one. The most capable generally available models in July 2026 cost roughly $10 per million input tokens and $50 per million output tokens, about double the top tier's price a year earlier. An organization that fixes its capability requirement sees falling costs. An organization that always chases the newest, most capable model sees rising costs.
The real driver is computation per task, not price per token.
Reasoning models generate substantially more billable output for the same answer. Retrieval systems bill for retrieved context on every call. Agent workflows resubmit accumulated context at every step, so cost compounds as steps increase. Gartner estimates agentic workloads consume five to thirty times the tokens of an equivalent single-turn interaction. This is how unit prices fall while invoices rise at the same time: the quantity of computation per task has been growing faster than the price per unit has been falling.
Tier selection matters more than any price movement will.
Published pricing spans more than a hundredfold between the cheapest and most expensive capability tiers. At 10 million processed documents a year, the same extraction task costs about £2,600 at the cheapest tier and £300,800 at the most expensive one (Appendix B). No price change in the past three years comes close to that range. Tier selection determines inference cost far more than market pricing does, and tier selection sits entirely within the buyer's control.
Fragmented systems make every AI initiative repeat the same cost.
Logistics groups built through acquisition typically run several warehouse and transport management systems, each with its own data model and its own definitions for core business terms. Every new AI initiative rebuilds the same integration and validation work instead of reusing prior work, which makes forward cost hard to predict. A shared data foundation reverses this pattern: the first initiative funds the setup cost, and every initiative afterward costs less.
Governed inference is a minor cost. Ungoverned, it dominates.
Modelled across 24 months, inference accounts for about 11 percent of annual program cost once routing, caching, and capacity planning are applied. Without those controls, inference rises to about a third of a total that is itself 2.8 times larger (Section 6).
The acquisition problem: many systems, no shared definitions
Contract logistics in the UK and Europe has consolidated substantially over the past decade. Scale in shared-user warehousing, density in transport networks, and the capital cost of automation have all rewarded larger operators, and consolidation of this kind proceeds acquisition by acquisition. This pattern is well known across the sector, from DSV's acquisition of Panalpina through GXO's acquisition of Wincanton to DSV's acquisition of Schenker.
Acquirers typically keep the operational systems of the businesses they buy, at least at first, and that choice is usually the right one. Warehouse and transport management systems are load-bearing for service delivery, and replacing them immediately after a transaction adds risk the business does not need to take on.
The cost accumulates anyway. Where several systems remain in place, the differences run deeper than the software itself.
- Event definitions differ. When a delivery counts as complete, what tolerance decides on-time, and how partial or refused deliveries are treated all vary by system. Group-level figures built from these sources need reconciliation, not simple aggregation.
- Master data differs. The same customer, site, carrier, or handling unit is identified differently in each system, so joining records across systems becomes probabilistic matching instead of a direct lookup, and that matching logic has to be maintained indefinitely.
- Units and hierarchies differ. Pallet, case, layer, and unit conventions are commonly inconsistent, which affects any measure expressed per unit of throughput.
- Integration work does not transfer. Event models, status codes, and interfaces differ by source, so connecting one system to an analytics or AI application teaches the next connection almost nothing, and the effort repeats in full.
- Validation effort repeats. Each source behaves differently, so each new application needs its own evaluation and supervised testing before it can be trusted.
This is what makes AI cost specifically hard to predict in this sector. A model conditions its output on the data it receives, so inconsistency at the source produces inconsistent answers that the model itself cannot detect. In practice, this surfaces as a disagreement between an AI-generated answer and an established report, which costs analyst time to resolve and erodes the confidence that adoption depends on. Tools that give business users direct query access are the most exposed to this problem: translating a plain-language question into a query requires governed definitions of the entities involved, and where a term resolves differently across systems, the result is a plausible wrong answer rather than an obvious error.
Data Hatch illustration. Directional shapes drawn from observed delivery patterns rather than measured data; no figure elsewhere in this article derives from this exhibit. The unified path costs more on the first use case, crosses over at the second, and falls from there. The crossover point depends on how much prior work each new application can reuse.
Without a shared foundation, every new AI initiative repeats this integration and validation cost from the start, so cost per initiative stays flat or climbs as more sources enter scope. With a shared foundation in place, the first initiative pays the setup cost and every initiative after it draws on that work instead of repeating it. Group reporting, network optimization, customer-facing data services, and any project spanning multiple divisions all benefit from the same foundation once it exists. Building it converts a cost the business would otherwise pay repeatedly into a cost it pays once. Section 7 sets out the practical routes to building it.
How AI cost actually moves
Total AI spend is the product of three factors: how many tasks are performed, how much computation each task consumes, and what that computation costs per unit. Most attention goes to the third factor, the price per token, but the first two account for nearly all of the volatility organizations actually experience, and design choices determine both of them.
Stanford HAI's AI Index tracks the cost of querying a model at a fixed benchmark score. That cost fell from $20 per million tokens in November 2022 to $0.07 by October 2024, a drop of more than 280-fold. Successive model generations have delivered more capability at equal or lower prices, and open-weight providers have added further downward pressure on top of that.
What falls is the price of a fixed capability. What rises is the price of the best available option, because each new generation adds a pricing tier above the previous ceiling. In July 2026, the most capable generally available models cost roughly $10 per million input tokens and $50 per million output tokens, up from $5 and $25 a year earlier. Output is priced at roughly five times input across providers; tier-by-tier rates appear in Appendix B.
| Mechanism | Effect on expenditure | Direction | In your control |
|---|---|---|---|
| Published rate for a fixed capability | Price per million tokens at a stated performance level | Falling | No |
| Position of the frontier | Each generation adds a tier above the last, at about twice the prior top rate | Rising | No |
| Capability tier selected | Routine work at the premium frontier can cost over 100 times the same work at the smallest sufficient tier | Sharply up if unmanaged | Yes |
| Reasoning output | Models that generate intermediate reasoning produce far more billable output per answer | Up | Yes |
| Multi-step agent architecture | Accumulated context is resubmitted at every step, so cost grows faster than the number of steps | Steeply up | Yes |
| Adoption and volume growth | Extending a proven application across the business multiplies the number of calls made | Up, by design | Yes |
Data Hatch analysis. Tier multiples are derived from published provider pricing as of July 2026 (Appendix B). The five-to-thirty-times estimate for agentic workloads is Gartner analysis published March 2026 (Appendix D, reference 2).
Data Hatch illustration, indexed to 100 at the start of the period. This is a mechanism, not a forecast: gradients are chosen for legibility and represent no specific provider.
A commitment expressed only as a rate gives limited protection, because it says nothing about the other two factors that determine total spend. A commitment expressed per unit of business work, such as cost per document processed or per query answered, constrains what the organization actually experiences and stays meaningful as volume grows. Section 5 sets out how to build a commitment of that kind.
Self-hosted versus purchased inference
Self-hosted inference is often proposed as the route to cost predictability, on the reasoning that infrastructure cost is fixed and known in advance and is not exposed to a provider's pricing changes. That reasoning holds only under specific conditions of scale and utilization, and those conditions rule out most operations.
Dedicated infrastructure is paid for whether or not it is actively working, so cost per unit of output depends heavily on utilization. Utilization gets most of the attention in this comparison, but it is not the binding constraint at the scale most enterprises operate. The binding constraint is engineering capacity: a fixed annual cost that does not fall as volume grows and has to be recovered before any per-token advantage appears in the total. At 80 percent utilization, infrastructure alone costs about $0.92 per million tokens. Adding 1.5 to 4 engineering full-time employees, depending on estate size, produces the all-in figures below.
| Accelerator units | Annual throughput at 80% | All-in cost per million tokens | Against small tier at $2.00 | Against frontier tier at $10.00 |
|---|---|---|---|---|
| 1 | 37.8 billion | $8.83 | 4.4× more expensive | 12% cheaper |
| 4 | 151 billion | $3.56 | 78% more expensive | 64% cheaper |
| 8 | 303 billion | $2.57 | 29% more expensive | 74% cheaper |
| 16 | 605 billion | $1.91 | 5% cheaper | 81% cheaper |
| 32 | 1,211 billion | $1.58 | 21% cheaper | 84% cheaper |
Data Hatch model. All-in cost includes infrastructure at published specialist-provider rates and engineering at UK fully loaded cost, assuming 80 percent sustained utilization. Purchased tiers are blended at 1,500 input and 500 output tokens per task from published pricing as of July 2026. Full derivation and sensitivities at Appendix A.
Against a small purchased tier, dedicated infrastructure does not become cheaper until sustained throughput passes roughly 500 billion tokens a year, equivalent to 16 or more accelerator units running at high utilization. Below that volume, the engineering overlay dominates the calculation and purchasing wins, in some cases by a factor of four.
Against the frontier tier, the comparison flips immediately. A single accelerator unit at high utilization already beats the frontier tier on cost, and the advantage widens with scale.
Data Hatch model based on published provider pricing as of July 2026, converted at $1.33 to £1. Per-document costs and full arithmetic at Appendix B.
A structured extraction task using about 1,500 input tokens and 500 output tokens shows the same pattern at the document level. At 10 million documents a year, the gap between the cheapest and most expensive tier is about 114-fold, close to £300,000 a year for identical work. That volume works out to about 20 billion tokens a year: half the capacity of a single accelerator unit, and two orders of magnitude below the point where dedicated infrastructure beats a small purchased tier. Purchasing costs about £30,100 a year at that volume, against an engineering overlay alone of £225,000 or more. The saving here comes from routing work to the right tier. Hosting the model in-house does not produce it at this volume.
Two conditions still justify self-hosting below the break-even point. Data residency requirements are satisfied by dedicated infrastructure by design. Protection from a provider's future pricing changes is what fixed infrastructure removes entirely. Measured against a frontier-tier alternative, both come at no extra cost. Measured against a small purchased tier, both come at a real premium.
Four controls that hold a cost envelope
Four controls account for most of the cost reduction that is actually achievable, and they mirror the practices the FinOps Foundation has formalized for AI cost governance specifically. Published research quantifies their effect, though the figures are specific to the benchmark tested, so the saving on any real workload has to be measured rather than assumed.
Route work to the smallest capability tier that can do the job.
Most enterprise workloads do not require the most capable model available. RouteLLM, from LMSYS and UC Berkeley, reports reductions above 85 percent on one benchmark while retaining about 95 percent of the reference model's performance, though the same method produces only 45 percent on a second benchmark and 35 percent on a third. FrugalGPT, from Stanford, reports savings up to 98 percent by cascading requests through progressively more capable models and stopping once an answer is good enough, though that figure comes from a single dataset of financial news headlines. Neither number belongs in a business case as a standalone planning figure. Together, they show that the mechanism works and that the achievable saving on a mixed workload most likely falls between 40 and 85 percent, pending measurement on the actual workload. This control works best when it is built into the platform as policy: requests are evaluated against a quality threshold and escalated only when that threshold is not met.
Cache repeated context, and shift deferrable work to batch processing.
Context that repeats across calls is billed every time unless it is cached, and providers typically discount cached reads by about 90 percent. That discount applies to reads only. Writes carry a premium and cache entries expire, so the blended saving depends on the ratio of reads to writes. Workloads with stable instructions and frequent calls approach the headline figure; workloads with varied context and infrequent reuse may see little benefit. Separately, work that does not need an immediate response can move to batch execution, typically discounted by about 50 percent with no similar caveat attached. Document extraction, classification, compliance review, and reconciliation are all schedulable this way.
Purchase committed capacity sized to measured base load.
Reserved capacity converts variable, consumption-based billing into a fixed periodic charge at a substantial discount. The risk is over-commitment: capacity that is reserved but not used is spend with no return. Size the reservation to measured base load, and leave peak demand on standard consumption pricing.
Negotiate contractual protection proportionate to the size of the commitment.
At enterprise scale, several protections are typically available to negotiate: rate protection for the term of the contract, so published price increases do not apply to committed volume; advance notice of model withdrawal or deprecation, so migration can be planned rather than forced; protection against changes in how usage is measured, since a metering change carries the same commercial effect as a price increase; data and workload portability, to preserve negotiating position at renewal; and periodic benchmarking against prevailing market rates.
| Control | Mechanism | Published effect, and its qualification |
|---|---|---|
| Capability tier routing | Policy-based selection with escalation by exception | 35 to 85 percent across three benchmarks in the routing study; up to 98 percent on a single dataset for cascading. Workload-specific; requires measurement. |
| Context caching | Retention and reuse of repeated prompt context between calls | About 90 percent on cache reads. Blended saving is lower, because writes carry a premium and entries expire. |
| Batch execution | Deferred processing for work not requiring an immediate response | About 50 percent against standard rates, applied consistently by major providers. |
| Committed capacity | Reserved or provisioned throughput sized to measured base load | Substantial discount against on-demand pricing. Realized benefit depends on the accuracy of the base-load measurement. |
Routing and cascading figures as published in Appendix D, references 3 and 4. Caching, batch, and committed-capacity terms as published by major providers as of July 2026, and should be confirmed at contracting.
These four controls reward early design. Retrofitting them later means reworking prompts, pipelines, and evaluation systems that were built without them in mind. Applied together, they support a commitment expressed as a maximum cost per unit of business work, held across an agreed volume range and reviewed on a set schedule. A rate-only commitment does not do either of those things.
A twenty-four-month cost model
The following model applies the mechanisms above to a representative program: 10 million processed documents and 2 million interactive queries a year by month 24. Three scenarios hold adoption and outcomes identical and vary only the controls applied, which isolates what the controls are worth. This is a designed comparison, not a measured one. The difference between scenarios is built into the model by construction, so it demonstrates the potential magnitude of the controls rather than proving that magnitude occurred in practice.
| Component | Optimized | Routed | Without controls |
|---|---|---|---|
| Document workload inference | £12,000 | £30,000 | £150,000 |
| Interactive workload inference | £14,000 | £26,000 | £301,000 |
| Data platform capacity and storage | £58,000 | £73,000 | £219,000 |
| Observability, evaluation, and compliance | £30,000 | £35,000 | £70,000 |
| Platform operation and support | £225,000 | £300,000 | £525,000 |
| Residual integration effort | £15,000 | £25,000 | £120,000 |
| Total annualized run rate | £354,000 | £489,000 | £1,385,000 |
| Inference as a share of total | 7% | 11% | 33% |
Data Hatch model. Composition of each case and derivation of every line at Appendix C. Modeled at an illustrative scale to establish relative magnitude and composition, not to forecast expenditure for any specific organization.
Inference is a minor cost once controls are applied.
At 11 percent of the routed case, inference costs less than platform operation, less than platform capacity, and about the same as observability and compliance combined. An organization that focuses only on token pricing is optimizing the fourth-largest line in its own budget. Remove the controls, and inference becomes the largest line after staffing.
Engineering capacity is the largest line in every scenario.
This is the single most important number in the model: one additional engineering full-time employee adds about £150,000 a year, roughly five times the entire document-inference line in the routed case. An approach that needs fewer people to operate beats an approach with lower licensing or inference cost but higher staffing needs. That favors reusing a shared foundation over repeating point-to-point integration work, and it favors purchased inference over dedicated hosting at these volumes.
The model is more sensitive to staffing than to any pricing assumption.
Removing an entire accelerator unit or changing the document-processing tier moves the total far less than adding one person does. Evaluate the operating model before the rate card.
Fixing the root cause: a common data foundation
Section 2 described how fragmentation compounds: every new AI initiative re-pays the same integration cost when systems and definitions are not shared. Four approaches address this, and they differ sharply in cost, timeline, disruption, and how much they actually improve cost predictability.
Operational consolidation
Migrating acquired businesses onto a single warehouse and transport management platform produces the most complete result, but it is a multi-year, high-disturbance program in its own right. It carries the cost of running both new and legacy systems in parallel for a year or more before any benefit appears.
Coexistence through middleware
Connecting existing systems with point-to-point interfaces, while leaving them in place, keeps disturbance low and resolves transactional problems well: an order raised in one system will appear correctly in another. It does not resolve the analytical problem, since the same data still means different things in each system, so reconciliation is still required for reporting.
Data foundation unification
Establishing a governed set of shared definitions over data that stays in its source systems is the fastest route to a consistent, group-wide view for reporting and AI, typically delivering initial value in three to nine months. It does not resolve operational duplication or transactional interoperability on its own, and it does not substitute for consolidation where consolidation is independently justified.
Selective consolidation
Combining the two approaches means systems are consolidated only where the business case for doing so stands on its own, and everything else is unified at the definition layer. Once shared definitions exist in one place, the cost of maintaining any remaining divergence becomes visible, which makes the case for each future consolidation measurable instead of assumed.
For most acquisition-built groups, data foundation unification is the practical starting point. It delivers the precondition for reliable group reporting and predictable AI economics without disturbing the operational systems the business depends on day to day, and it can proceed alongside a longer-term consolidation decision rather than waiting on one.
Platform selection, and the build-versus-buy question
Once the choice of layer is settled, platform selection becomes more straightforward. A well-specified definition layer can run on a major vendor's integrated data platform, an open lakehouse, or a managed analytical service, and any of the three can deliver the result. A poorly specified one will be expensive on all three, because cost is set by the work of agreeing what the definitions actually are, not by the technology that enforces them.
AI-assisted development has narrowed the traditional cost gap between building this layer and buying a packaged version of it. That gap alone should not decide the question. Section 6 showed that engineering capacity, not licensing or inference cost, is the largest line in every scenario. The real question is which option needs fewer people to run at steady state, and how far each option locks in pricing that cannot be renegotiated later. Apply that test to a custom build with the same rigor as to a vendor platform.
Putting this into practice
Three practical points follow from the model above.
Do not fix a cost ceiling before measuring your own baseline.
The figures in this article are illustrative, built to show the relative size and behavior of each cost component, not to forecast any specific organization's spend. A defensible cost commitment depends on measured document volumes, the actual token profile of real documents, and the routing distribution achievable on real workloads, none of which can be known from outside the operation. A number offered before that measurement is either padded against uncertainty or unreliable.
Start with a short, bounded baseline stage before committing to the wider program.
A baseline stage of four to six weeks, focused on inventorying source systems, measuring real document and query volumes, and capturing current cost and effort for the processes being replaced, produces the inputs a real commitment depends on. This stage should be short enough, and priced separately enough, that it requires no commitment to the larger program.
Sequence the shared data foundation ahead of or alongside the first AI workload, not after it.
The first definition domain, covering the terms used most widely across the business, typically takes eight to twelve weeks to establish and is what makes group-level figures reconcile without manual adjustment. The first production workload should be the highest-volume document process, implemented with routing, caching, and batch execution from day one, since retrofitting these controls later costs more than building them in from the start.
Cost predictability, in the end, is not a separate initiative from the AI program itself. It is what happens when the four controls in Section 5 and the shared foundation in Section 7 are treated as day-one architecture rather than a later cleanup project.
Appendices
ABreak-even model for dedicated inferenceOpen +Close −
This model computes an effective cost per million tokens for inference on dedicated infrastructure, including engineering capacity, and compares it against published per-token pricing on the same blended basis.
Infrastructure cost per million tokens equals the accelerator's hourly rate, divided by throughput multiplied by 3,600 and by utilization, multiplied by 1,000,000.
All-in cost per million tokens equals annual infrastructure cost plus annual engineering cost, divided by annual throughput in millions of tokens.
Inputs
| Input | Value applied | Basis |
|---|---|---|
| Accelerator hourly rate | $3.99 | Published list pricing from specialist infrastructure providers, July 2026. Rates in this class span roughly $1.50 to $6.00 depending on generation and provider, and equivalent capacity from major cloud providers is priced higher. |
| Sustained aggregate throughput | 1,500 tokens per second | Conservative planning figure for a mid-size open-weight model on a single accelerator, based on published open-serving benchmarks. A frontier-scale open-weight model requires several accelerators, multiplying infrastructure cost accordingly. |
| Hours per year | 8,760 | Continuous availability. |
| Engineering capacity | 1.5 to 4 full-time employees | £150,000 each at UK fully loaded cost, covering capacity planning, model upgrades, security maintenance, evaluation, and incident response. Scales with estate size rather than volume. |
Infrastructure pricing and serving throughput both move within single quarters and should be re-confirmed at the point of any decision (Appendix D, reference 7).
Known limitations
- Throughput is applied uniformly to input and output tokens. Processing input is faster than generating output, so applying a generation-speed figure to input overstates compute time, and therefore cost, for dedicated infrastructure. This makes the model conservative against self-hosting.
- Several infrastructure costs are omitted, including networking, storage, redundancy and failover, idle capacity held for peak demand, model licensing, and infrastructure for fine-tuning, distillation, and evaluation. Including them would raise the dedicated-hosting figures, working in the opposite direction from the point above.
- Utilization of 80 percent is favorable. It is achievable for continuous batch processing but not for interactive workloads. At 30 percent utilization, the infrastructure component rises from about $0.92 to about $2.47 per million tokens before engineering cost is added.
Computed results
The infrastructure component is about $0.92 per million tokens at 80 percent utilization, $1.48 at 50 percent, $2.47 at 30 percent, and $3.70 at 20 percent. It is inversely proportional to utilization, so each halving of utilization roughly doubles it.
Adding engineering capacity produces the all-in figures in Exhibit 4. At one accelerator unit, the all-in cost is about $8.83 per million tokens, of which infrastructure makes up about 10 percent. At 32 units, it falls to about $1.58, of which infrastructure makes up about 58 percent. Dedicated infrastructure undercuts a small purchased tier blended at $2.00 between 8 and 16 units, or roughly 400 to 600 billion tokens a year.
BMethod for estimating document workload costOpen +Close −
The volume used throughout this article, 10 million documents a year, is illustrative: it represents the order of magnitude of document flow in a large contract logistics operation. Substituting measured volumes reproduces the calculation for any organization.
Per-document cost at each capability tier
| Capability tier | Indicative published rate per million tokens | Cost per document | At 10 million documents |
|---|---|---|---|
| Budget | $0.10 input, $0.40 output | $0.00035 | £2,632 |
| Small | $1.00 input, $5.00 output | $0.00400 | £30,075 |
| Mid | $3.00 input, $15.00 output | $0.01200 | £90,226 |
| Frontier | $5.00 input, $25.00 output | $0.02000 | £150,376 |
| Premium frontier | $10.00 input, $50.00 output | $0.04000 | £300,752 |
Assumes 1,500 input and 500 output tokens per document, representative of a structured extraction on a typical business document, converted at $1.33 to £1. Rates are indicative of published tier pricing as of July 2026 and vary by provider (Appendix D, reference 8).
Worked calculation
At the small tier, $1.00 per million applied to 1,500 input tokens gives $0.0015, and $5.00 per million applied to 500 output tokens gives $0.0025, for a total of $0.0040 per document, or $40,000 at 10 million documents, about £30,075. The same calculation at the premium frontier tier gives $0.0400 per document and $400,000 annually, about £300,752, a factor of about 114 between the two tiers.
Caching the input portion, where the extraction instruction is common across documents, and submitting the work for batch execution both reduce these figures further. Both apply at every tier, so neither changes the relationship between them.
Blended rates used in the hosting comparison
A self-hosted deployment is billed for total tokens processed rather than separately for input and output. At the 1,500-and-500 token profile, the blended rate is $0.175 per million tokens at the budget tier, $2.00 at small, $6.00 at mid, $10.00 at frontier, and $20.00 at premium frontier.
Volume estimation
Where measured counts are unavailable, volume can be estimated from operational throughput. A contract logistics operation typically generates a dispatch instruction, a consignment or delivery note, a proof of delivery, and an invoice per shipment, plus customs documentation on cross-border movements and claims or compliance records on a portion of activity. A multiplier of four to six documents per shipment is a reasonable starting assumption until measured counts are available, though the estimate is sensitive to how the operational unit is defined.
CComposition of the scenario casesOpen +Close −
Three cases model an identical program reaching identical adoption at month 24, differentiated only by the controls applied, processing 10 million documents and 2 million interactive queries annually at that point.
Governance variables applied
| Variable | Optimized | Routed | Without controls |
|---|---|---|---|
| Tier used for document work | Small, distilled | Small | Frontier |
| Tier used for interactive work | Small and mid | Mid | Frontier |
| Context caching | Applied by default | Applied to major workloads | Not implemented |
| Batch execution | Consolidated windows | Applied where available | Not applied |
| Agent step and budget limits | Enforced with budgets | Enforced | Not enforced |
| Platform capacity sizing | Right-sized to base load | Reserved to base load | Provisioned to peak |
Derivation of each line
| Line | Optimized | Routed | Without controls |
|---|---|---|---|
| Document workload inference | Distilled, tuned model plus batch execution, about 60% off: £12,000 | Small tier at 10 million documents (Appendix B): £30,075 | Frontier tier: £150,376 |
| Interactive workload inference | Majority routed to small tier at 85% cache hit: £14,000 | Mid tier, 60% cache hit. 4.8bn cached input at $0.30 plus 3.2bn uncached at $3.00 equals $11,040; 1.6bn output at $15.00 equals $24,000; total $35,040: £26,000 | Frontier tier, no caching, 5× token multiplier for agents without step limits (lower half of reference 2). 40bn input at $5.00 plus 8bn output at $25.00 equals $400,000: £301,000 |
| Data platform capacity and storage | Tighter sizing: £58,000 | £45,000 production under annual reservation, £15,000 non-production, £13,000 storage (about 50TB): £73,000 | Provisioned to observed peak at about 3× base load: £219,000 |
| Observability, evaluation, and compliance | £30,000 | Monitoring, evaluation harnesses, guardrails, logging, and record-keeping: £35,000 | £70,000, reflecting reconciliation of inconsistent outputs |
| Platform operation and support | 1.5 FTE: £225,000 | 2 FTE at £150,000 fully loaded: £300,000 | 3.5 FTE: £525,000, reflecting incident load and reconciliation effort |
| Residual integration effort | Never zero: source systems change, interfaces are revised, new sources enter scope: £15,000 | £25,000 | No shared layer, so integration is re-incurred across sources: £120,000 |
Capacity pricing varies widely between platforms, so the platform capacity line is the most sensitive to platform choice and should be re-confirmed against the actual vendor rate card.
Totals and sensitivity
The three cases total about £354,000, £489,000, and £1,385,000, a ratio of about 2.8 times between the routed case and the case without controls. Engineering headcount is the most influential assumption in the model, since one full-time employee moves any case by £150,000. Platform capacity is the second most influential, varying threefold across the cases and depending on a vendor rate card that should be confirmed before use.
DReferencesOpen +Close −
Listed in the order first relied upon. Pricing and infrastructure references are point-in-time, established July 2026.
- Stanford Institute for Human-Centered AI, AI Index Report 2025, research and development chapter, for the recorded fall in inference cost at fixed benchmark performance between November 2022 and October 2024.
- Gartner, analysis published March 2026, estimating that agentic workloads consume 5 to 30 times the tokens of an equivalent single-turn interaction. Widely reported; the primary Gartner publication should be obtained before the figure is relied upon contractually.
- Chen, L., Zaharia, M., and Zou, J., FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance, arXiv preprint 2305.05176, May 2023. The headline reduction is reported on a financial news headline dataset.
- Ong, I., Almahairi, A., Wu, V., Chiang, W., Wu, T., Gonzalez, J. E., Kadous, M. W., and Stoica, I., RouteLLM: Learning to Route LLMs with Preference Data, arXiv preprint 2406.18665, June 2024. Reported reductions differ substantially across the three benchmarks evaluated.
- FinOps Foundation, FinOps Framework, including the FinOps for AI technology category and the FinOps Open Cost and Usage Specification.
- vLLM project, published performance benchmarks for open-weight model serving.
- Published list pricing for accelerated compute from specialist cloud infrastructure providers, retrieved July 2026.
- Published application programming interface pricing from major model providers, including tier rates, cached input discounts, and batch execution discounts, retrieved July 2026.
References 7 and 8 are point-in-time observations. Figures derived from them establish relative magnitude, between tiers and between hosting options, which has been stable. They do not fix absolute cost, which has not been stable, so absolute figures should be re-confirmed before any commitment. Reference 2 is an analyst estimate reported at second hand, used only to characterize the direction and approximate scale of agentic token consumption; Appendix C applies the lower half of the reported range.
ETerms usedOpen +Close −
| Term | Meaning as used in this article |
|---|---|
| Capability tier | A band of model capability and its associated price. This article refers to tiers rather than named products, since product names change faster than the underlying economics. |
| Frontier | The most capable tier generally available at a given moment. Its position moves upward with each model generation, and its price has risen accordingly. |
| Dedicated inference | Operation of models on owned or reserved infrastructure, instead of purchasing output from a provider on a per-token basis. |
| Utilization | The proportion of available infrastructure capacity actually performing work. A key variable in dedicated inference economics, though not the binding one at moderate scale. |
| Routing | Direction of each request to the least capable model sufficient to perform it, escalating only where a defined quality threshold is not met. |
| Context caching | Retention of repeated prompt content between calls, so it is not billed at the full input rate each time. Reads are discounted; writes generally carry a premium. |
| Definition layer | A governed set of business definitions applied over data that stays in its source systems, so a given term resolves consistently across an organization. |
| Unit cost | Cost expressed per unit of business work, such as per document processed or per query answered, rather than per token consumed. |