FinOps for AI: put the whole service behind the cost per outcome

A cheaper model does not automatically make an AI service cheaper to run. If the replacement needs more retrieved material, more attempts or more human correction, the saving can disappear outside the inference invoice. For the FinOps lead, the job is to connect those costs to the same business outcome.

That requires a service account rather than a model leaderboard. It should show what the organisation spends to deliver accepted work, who can change each cost driver and which apparent savings are simply work moving elsewhere. Token prices remain useful, but they answer a narrower question.

Decide what counts as an outcome

Choose the unit with the business owner. For a customer-support assistant, it might be a case resolved to an agreed standard without reopening during a defined review period. For document extraction, it might be a record accepted by the downstream process after any required correction. Neither is the same as a generated response.

The FinOps Foundation’s unit-economics guidance describes moving from token measures towards outcomes such as assists or cases deflected. The difficult part is making the chosen denominator trustworthy. Agree the acceptance rule, the measurement period and the population before using the metric to compare versions.

Count failed attempts and retries in the numerator even when they produce no accepted outcome. Count a task once when it eventually succeeds, rather than treating each regeneration as new business value. Keep difficult and routine work distinguishable so that routing harder cases to humans does not make the automated service appear to improve by changing its workload.

A falling cost per outcome can also conceal a narrower service. Report the share of incoming work the system accepts and completes, alongside its unit cost and quality. Otherwise the easiest route to a good number may be refusing expensive cases that the business still has to handle.

Follow the costs beyond the model call

Build the account from the workflow. Include source-data preparation, ingestion, embeddings, retrieval, model inference, orchestration, storage, monitoring, evaluation and human review where those activities support the service. Separate one-off implementation from recurring operation, while keeping both visible in the investment case.

Named billing mechanisms make this more than a theoretical concern. The Amazon Bedrock pricing documentation describes query-based charges for rerank models in its self-managed Knowledge Bases section. It also identifies judge-model token charges for LLM-based evaluation and regular Knowledge Bases usage charges when evaluating retrieval-augmented generation. The model generating the user-facing answer is therefore not the only activity to budget for.

Evaluation can change another cost category too. LangSmith documents that running an online evaluator upgrades the relevant trace to extended retention, affecting trace pricing. When an engineering team increases evaluation coverage, the forecast should account for the observability consequence as well as the evaluator’s model use.

Do not assume that every listed component is separately charged in every product. Some services bundle functions; others meter them individually. Reconcile the architecture with the purchased plan and the bill, then identify internal costs that the supplier cannot show.

When a token saving makes the service more expensive

Consider this hypothetical monthly account, using illustrative GBP figures rather than supplier prices. Both versions deliver 8,000 accepted outcomes under the same acceptance rule.

Monthly cost	Original service	Changed service
Inference	£8,000	£4,000
Retrieval and data preparation	£3,000	£5,000
Monitoring and evaluation	£2,000	£3,000
Human review and correction	£7,000	£10,000
Total	£20,000	£22,000
Cost per accepted outcome	£2.50	£2.75

Inference spending halves, but the total rises by 10%. The review should investigate whether the additional retrieval and correction are caused by the change, by different work entering the service or by another operational issue. The table identifies the commercial consequence; it does not prove causation.

If reviewers are salaried staff, the allocated labour cost is not necessarily additional cash leaving the organisation that month. It still represents capacity consumed. Report that distinction so finance can separate cash savings, avoided future hiring and time available for other work. Do not claim a headcount saving merely because a model completed some tasks faster.

Allocate shared costs without hiding unused capacity

A shared retrieval platform or evaluation team may serve several AI products. Choose an allocation basis that reflects use where practical: indexed data volume, retrieval activity, review time or another observable driver. Keep the unallocated balance visible and explain why a proxy is being used.

Show direct variable cost alongside the fuller service cost. The former helps engineers understand what another unit of work consumes; the latter helps the business judge sustainability. Blending the two into one unexplained number creates arguments about whether a service is improving whenever platform overhead changes.

Unused committed capacity also needs an owner. Distributing it invisibly across successful requests can make product teams appear inefficient without showing the purchasing decision that created the cost. Equally, excluding it entirely can make a service appear cheaper than the organisation’s actual commitment.

Make the variance lead to an action

Break the monthly change into demand, workload mix, unit consumption, rates, quality and shared-cost allocation. Assign the resulting actions: engineering can investigate retries; the data team can reduce unnecessary re-indexing; the product owner can change acceptance or review design; procurement can examine commitments. FinOps should make those trade-offs comparable and verify the result.

This is the AI version of a broader problem: a smaller cloud bill can hide a more expensive service. Once the full account is in place, use a workload-based inference comparison to examine model costs within it. That preserves the value of token-price analysis without asking it to stand in for the whole business case.