AI token economics: what does a successful task actually cost?

Revised October 18, 2026.

A cheaper model can leave you with a more expensive workflow. When it needs more attempts, misses information or sends work back to a person, the saving on the API bill can disappear before the task is finished.

Consider an illustrative supplier-onboarding assistant. It prepares a summary quickly, but the reviewer has to reopen three documents because the citations concern another legal entity. The model call succeeded. The review did not. Downloadable weights could give the organization a different inference operator; that choice would still leave the entity-resolution failure to be fixed.

When I compare AI systems, I want to know what it costs to produce a result someone can use. That requires a common acceptance rule, a view of the work around inference, and a decision about where the system may operate. An API, a local specialist and a hybrid can each be sensible choices. Their economics depend on the task and the controls they must support.

Start with the result someone will accept

For the supplier review, acceptance might require the correct entity, source-supported fields, current policy, identified missing evidence, and no unauthorized change to the supplier record. Define those conditions before testing models. A person can approve the result without the system having completed the work autonomously; report those operating modes separately.

For a fixed batch, count original tasks, accepted results, rejections, escalations and pending cases. A task that required five calls still contributes one result to the count, with the cost of all five calls attached. A correct escalation can be an accepted outcome if the task was to identify cases requiring review; define it separately from completing onboarding.

The FinOps Foundation’s unit-economics guidance connects technology costs with business units. Here, tokens describe resource consumption, while cost per accepted review describes what that consumption bought. The basic calculation is:

cost per accepted task =
  (model + tools + allocated platform + review + rework costs)
  / accepted original tasks

Keep the boundary visible. If downstream manual completion is excluded, the result describes the evaluated batch, not the entire service. Zero accepted tasks means there is no finite cost per accepted task to report.

Calculate the bill without counting the same tokens twice

Use the provider’s billable categories and usage records. They may distinguish ordinary input, cache writes, cache reads and output. Search, execution, storage, media and external tools can add charges. A text-only estimate cannot describe a voice workflow.

For a hypothetical call, assume 30,000 uncached input tokens at $2 per million, 70,000 cache-read tokens at $0.20 per million, and 3,000 billable output tokens at $8 per million:

(30,000 × $2 + 70,000 × $0.20 + 3,000 × $8) / 1,000,000
= $0.060 + $0.014 + $0.024
= $0.098

These are teaching assumptions, not a current provider quotation. The example uses three disjoint billing categories and assumes no separately priced cache writes. Where a provider prices cache creation, assign those tokens to its write category at the applicable rate instead of counting them again as ordinary input. Add non-token tool charges separately. OpenAI’s reasoning documentation states that reasoning tokens are billed as output tokens. If the billable output total already includes them, adding them again overstates cost.

Caching also needs actual usage evidence. A policy document may appear in every review, but reuse depends on the stable prefix, retention and the provider’s rules for that model. The current OpenAI caching guide describes its caching behavior; Claude’s caching documentation distinguishes cache creation, cache reads and ordinary input. Use the categories the chosen service actually reports and avoid overlapping counts. That makes a failed reuse assumption visible when the production bill arrives.

A higher API bill can buy a cheaper workflow

Imagine two configurations processing the same 1,000 supplier reviews under identical acceptance rules. All figures are illustrative. Review labor is valued at $60 per hour; that valuation does not mean those hours can immediately be removed from payroll.

Batch cost or outcome	Configuration A	Configuration B
Model calls	$60	$150
Tools	$30	$40
Allocated platform	$40	$60
Review and rework	30 hours / $1,800	15 hours / $900
Total batch cost	$1,930	$1,150
Accepted original tasks	850	920
Cost per accepted task	$2.27	$1.25

B’s model bill is two and a half times A’s, yet its accepted result costs less. Costs incurred on the 150 unaccepted A cases and 80 unaccepted B cases remain in the totals. Show any later manual completion separately.

The example does not select a model size. A smaller model with a bounded job and reliable context could produce a better result. It selects a way to compare configurations: keep the acceptance rule and task difficulty stable, then include the work needed to reach that standard.

Downloadable weights change the deployment options

An API supplies inference as a service. Downloadable weights let an organization operate trained parameters on infrastructure it controls, subject to the release’s terms. A managed hosting service can operate those same weights for a customer. Weight availability, operator and hosting location are separate decisions.

Open weights and open source also deserve separate entries in the review. The OSI Open Source AI Definition 1.0 requires data information, complete training and running code, and parameters in the preferred form for modification. Its data information includes descriptions of all training data, including unshareable data, and listings of public and third-party sources. A permissive license on weights alone does not establish that the complete system meets that definition.

Read the particular release’s license before assigning economic value to customization or redistribution. Microsoft’s Phi-4-mini-instruct card lists MIT. The original Llama 3 agreement grants conditional permissions and imposes attribution, use and other obligations. It is a dated example, not a description of every later Llama release. Record the checkpoint, revision and terms together.

For an internal service, the question may be whether the intended processing is permitted. For an appliance shipped to customers, packaging and redistribution enter the decision too. Include the runtime, tokenizer, adapters and document rights in that review. The model’s download price does not settle them.

Give a small model a job it can earn

The supplier workflow contains several different jobs: extracting a registration number, matching the entity, interpreting an exception and drafting a recommendation. Running all four through the same general model can obscure where capability is actually needed.

A local specialist might extract a small set of fields from familiar document types, with deterministic checks for formats and cross-document agreement. The exceptions can go elsewhere. The Phi-4-mini-instruct card describes a 3.8-billion-parameter model and also warns about factual-knowledge limitations. Those are useful properties to investigate; its developer-reported benchmarks do not establish fitness for supplier onboarding.

Build the comparison around documents held out from tuning, including unfamiliar layouts, missing identifiers and contradictory evidence. Compare exact fields, unsupported values, escalation and review effort against the existing extraction method and a capable API configuration. A smaller model should pass the minimum quality threshold before its operating cost becomes a reason to choose it.

It is tempting to describe a highly capable specialist as “narrow superintelligence”. That claim needs more than a win against one convenient baseline: it needs strong relevant comparators, held-out and difficult cases, error criteria, and evidence that the advantage survives operational variation. I would report the bounded result first. Even exceptional field extraction conveys no authority to approve the supplier or release a payment.

Self-hosting moves costs into your operating model

The self-hosted bill includes capacity, even while requests are absent. Purchased hardware needs an allocation over its useful life, with financing or opportunity cost where relevant; rented capacity brings reservation commitments and a scale-down policy. An accelerator that sits idle overnight may still be entirely reasonable for the service. Its cost must survive the calculation. CPU, memory, storage, networking and recovery capacity belong beside it.

Utilization must be measured against useful work at the required latency. A busy accelerator that produces unacceptable answers has not achieved the business objective. A server tuned for maximum batch throughput may make an interactive reviewer wait. The vLLM metrics documentation separates queue time, time to first token, output timing and end-to-end latency; each exposes a different constraint.

Record the model, precision, runtime, hardware, prompt lengths and concurrency. Quantization can change memory needs and behavior; test the deployed representation. Weight-file size excludes memory used by activations, caches and runtime buffers. The measured service, including its busiest periods, must fit the capacity plan.

Serving optimizations have boundaries too. vLLM’s prefix-caching documentation says reuse reduces prefill work, not the decoding of new output. Repeated long instructions may benefit; long generated reports still require generation. Test the actual request distribution instead of transferring a throughput claim from another workload.

A break-even calculation needs volume and quality

Take the bounded field-extraction step, not the whole supplier review. The following monthly assumptions are hypothetical dollar amounts, not cloud or hardware quotations. Assume one serving request per submitted task, identical task mix, 95% accepted results, and the same necessary review effort.

The API option has $100 of fixed platform cost and $0.12 of serving cost per task. The local option has a fixed cost of $1,200: $800 of capacity allocation and $400 of allocated operations, security, evaluation and update work. Its request-variable serving cost is assumed to be $0.02. Both add $0.20 in review costs per submitted task, equivalent to twelve seconds at $60 per hour. Include energy within the capacity allocation here; do not add it again.

API monthly cost   = $100 + N × ($0.12 + $0.20)
Local monthly cost = $1,200 + N × ($0.02 + $0.20)
Accepted tasks     = 0.95 × N

Equal-cost volume  = ($1,200 − $100) / ($0.32 − $0.22)
                  = 11,000 submitted tasks per month

Submitted tasks / month	API total	Local total	API / accepted task	Local / accepted task
5,000	$1,700	$2,300	$0.36	$0.48
15,000	$4,900	$4,500	$0.34	$0.32

This is a recurring operating-cost comparison. It excludes one-time data preparation, tuning, integration and initial evaluation. For an investment decision, add those costs over an explicit evaluation horizon, together with migration and exit costs. The 11,000-task crossing is not an investment-payback calculation.

The crossing exists only under those assumptions and within available capacity. If 11,000 requests require another server, revise the fixed cost. If the local model adds ten seconds of review per submitted task on average, that adds about $0.167 each at the assumed rate, exceeding the $0.10 serving advantage. There is then no break-even in this simple comparison with the API option unchanged.

Use different acceptance rates when results justify them; divide each total by its own accepted count. Include fallback calls and their review. APIs can remain cheaper for irregular demand, low volume, short-lived pilots or workloads where the local option needs substantial repair. Existing hardware can change the incremental decision, but its use still consumes capacity that another workload might need.

Sovereignty begins with the data path

An organization may require local operation even when an API has the lower cost. State the requirement before comparing prices: which information must remain where, who may operate the service, and what control must survive a supplier change? A model developer’s nationality answers only part of that inquiry.

Trace the supplier documents through upload, extraction, inference, retrieval, logs, backups and support. Identify where processing occurs, where data persists, who can administer it, and whether telemetry or remote updates cross the intended boundary. A locally running model can still sit inside an application that exports prompts or diagnostic records.

Provider controls also differ. The current OpenAI data-controls documentation distinguishes storage residency from regional inference processing, limits support by endpoint and model, and excludes system data from residency coverage. Inspect the applicable service configuration and terms rather than infer an inference-location guarantee from a regional address.

Key control is another specific requirement. Microsoft’s Azure OpenAI encryption documentation for Foundry classic describes customer-managed keys for data at rest in that service. A review can therefore record who controls those keys while separately establishing the network paths, administrative access and inference arrangements. An encryption setting answers one question in that review. It cannot stand in for the rest of the data-flow analysis.

Model provenance and licenses remain relevant even when processing is isolated. So do supply continuity, update rights, audit evidence and the ability to recover or migrate. Reject an option that cannot meet a mandatory boundary. Compare the cost of the options still eligible, including the operational work needed to maintain that boundary.

Security, updates and energy belong in the estimate

A downloadable package becomes part of the software supply chain. It can contain custom loading code and dependencies as well as tensors. Hugging Face’s pickle-security documentation explains how untrusted serialized objects can execute code when loaded. Budget artifact inspection, controlled loading, dependency maintenance and incident response; a familiar repository name is insufficient evidence.

After loading, separate inference from authority to act. A model that extracts a bank account should not acquire permission to replace the approved payment destination. Controls at the receiving system, access logs and tests of denied actions take work. Include their cost in both API and local designs, allocating only genuinely different costs to the comparison.

Updates create another recurring expense. When a new checkpoint extracts an identifier differently, the application owner has to determine whether the change fixes an error or breaks a useful behavior. Runtime, adapter and policy changes deserve the same comparison against a retained regression set. Fund that work and record the versions used. Buying inference from a provider leaves the application owner with this evaluation responsibility too.

Measure energy at a declared boundary. The MLCommons Power working group develops methods combining benchmark performance, power and energy. For a service estimate, identify idle and active consumption, supporting equipment and electricity cost, taking care that a cloud bill may already include energy while a local meter may omit cooling. A carbon-saving claim needs its own workload and accounting evidence. Weight access and model size cannot establish that conclusion.

Route by evidence, with a permitted fallback

A hybrid can keep routine extraction local while a more capable service handles difficult interpretation. Anthropic’s routing guidance describes classification into distinct tasks and emphasizes accurate classification. That is a design pattern, not a measured saving for this workflow.

Start with observable routing signals: supported document type, available identifiers, validated fields and contradictions. A model’s confidence statement alone is a weak basis for promotion. Define which failures trigger another attempt, which require a stronger model, and which must stop for missing evidence or authority. A stronger model cannot supply a purchase order that does not exist.

For an illustrative 1,000-task batch, a $0.01 first pass costs $10. If 20% then use a $0.10 fallback, those calls add $20. The $30 serving total excludes routing, tools and review; sending every task directly to the $0.10 option would cost $100. Test whether the routed configuration meets the same quality threshold before describing the difference as useful savings.

Fallback must respect the original data boundary. If confidential documents cannot leave the local environment, an external API is not an available escape route. Another local model, an authorized reviewer or a deferred case may be the appropriate outcome. Limit repeated attempts, record why routing occurred, and charge the full sequence to the original task.

Keep the experiment small enough to inspect

After tuning, return to held-out tasks containing the failures the service will need to handle: stale policies, ambiguous entities, missing pages, tool failures and denied permissions. Inspect the resulting records as well as the responses. Anthropic’s agent-evaluation guidance makes this distinction between an interaction and its resulting state; a polished recommendation cannot establish that the intended record was updated correctly.

Break results down by task type. Report acceptance, review minutes, latency, escalation and policy failures together. A favorable average can hide a poor exception path. For consequential workflows, minimum quality and authority requirements must be met before comparing eligible configurations on cost.

The editable cost workbook compares two AI workflows with a manual baseline. Its four sheets cover model routing and retries, token billing, external APIs and tools, human effort, operating costs, implementation allocation, and volume and break-even analysis. Replace the illustrative assumptions with your own rates and measured task outcomes.

I would leave the review with one bounded decision: use the API for now, fund a local comparison, change the routing rule, or stop until a recurring failure is understood. Released time is capacity; it becomes a financial saving only when the operating model turns it into one. The useful number is the cost of accepted work under controls the organization can actually maintain.

Related reading

Enterprise AI in 2027: seven decisions to make before you scale

Decision Velocity: examine the complete decision workflow

Share or cite this article

LinkedInEmail

Ajith Vallath Prabhakar. “AI token economics: what does a successful task actually cost?” ajithp.com, 17 October 2026. https://ajithp.com/2026/10/17/ai-token-economics/