AI cost surprises usually come from two predictable failures: teams ship with the development model instead of the production model, and they underestimate how often the workflow will run once it is embedded into the business. The fix is not a larger budget after the invoice arrives. It is a pre-deployment cost audit that maps every inference call, matches model capability to task complexity, validates lower-cost alternatives, and models production volume before the architecture is committed.
The pattern is consistent enough to be predictable. An engineering team builds an AI workflow in development, running it against a frontier model because that combination works — fewer edge cases, faster iteration, quicker path to something the team can demo.
The workflow ships.
Two or three billing cycles later, someone pulls the invoice and asks why this feature costs $18,000 a month.
The task hasn't changed. The model hasn't changed. What changed is that 10 calls a day became 1,200 calls a day, and the development model was never reconsidered for production. “Simple Python tasks are blowing up AI costs” is how one CIO publication described it — engineering teams discovering that the model selected in sprint one is consuming infrastructure spend that exceeds the product's revenue. 1
Two failure modes account for nearly every AI cost surprise in production. Both are predictable. Both are almost never caught before deployment.
Failure mode 1: task-model mismatch
Development defaults to frontier models for good reasons. They handle ambiguity better, require less prompt engineering to produce consistent output, and compress the iteration cycle.
The problem is that nothing in the development process forces a reconsideration before production. The model that worked in the pilot is the model that ships.
What that costs depends entirely on what the task actually requires.
Most enterprise AI workflows break down into a small set of operation types: classification, routing, extraction, and structured summarization. These tasks require pattern recognition and context assembly. They do not require deep reasoning.
For well-defined tasks with clean inputs, smaller models can often achieve performance that is functionally equivalent for the business use case — not theoretically identical, but indistinguishable in outcome where it matters: accuracy on the specific domain corpus, output format compliance, and business acceptance thresholds.
The cost differential between model tiers on these tasks is currently 3× to 30×, depending on the specific models compared. A frontier model in the $5–$15 per million input tokens range may perform the same document classification as a task-appropriate small model in the $0.15–$0.50 per million tokens range.
At 100,000 classifications per day, with an average of 500 input tokens and 100 output tokens per call, that differential can become roughly $30,000 per month versus $1,000–$3,000 per month on the same workload — depending on the providers and tiers selected.
The FinOps visibility problem compounds this. Many organizations can report aggregate cloud spend, but cannot reliably attribute inference cost to a specific product feature, workflow, customer segment, or business process. 2
At 1,000 calls per day, the cost difference between model tiers is sub-visible against overall cloud spend — easy to defer. By the time the workflow reaches 50,000 or 100,000 daily calls, the model choice is embedded in downstream systems that nobody wants to touch.
The conversation about cost happens at the worst possible moment.
For each inference call the workflow makes, the question to ask is simple:
What does this call actually require?
Pattern recognition
Classification, routing, binary decisions, and field extraction from structured inputs. Small models handle well-defined versions of these tasks effectively. No frontier model advantage justifies a 10–30× cost premium on tasks where the input space is constrained and the output format is fixed.
Context assembly
Summarization, reformatting, and template-filling from unstructured inputs. Mid-tier models generally perform here. Reserve frontier capability for tasks where assembly requires judgment across ambiguous or contradictory inputs.
Reasoning and synthesis
Complex analysis, multi-step judgment, and generating novel output from loosely defined inputs. Frontier models earn their cost here. If the task would require careful thought from a competent human, it likely requires frontier capability.
Agentic or long-horizon tasks
Autonomous workflows, tool use, and multi-step planning. Frontier models with appropriate context management are more defensible here. Cost-per-task will be higher; model capability justifies it if the task genuinely requires it.
One caveat before acting on this classification: switching models requires validation, not just classification.
You cannot confirm that a smaller model performs at an acceptable level without testing it against a representative corpus of real inputs — a golden dataset with defined accuracy thresholds and business acceptance criteria. The classification framework tells you where to look for savings; the eval tells you whether you found them.
Skipping the eval and shipping based on classification alone creates a different kind of production incident.
Failure mode 2: production volume underestimation
For interactive workflows triggered by user actions, pilots typically run at a fraction of production volume. This is understood in principle and ignored in practice.
The reason it gets ignored is simple: at pilot volume, the cost is so low it doesn't register as a variable worth modeling.
A workflow making 15 API calls per day at $0.002 per call costs $0.03/day. That number doesn't prompt a cost conversation.
The conversation that needs to happen — “what does this cost at 2,000 calls per day?” — doesn't happen because nobody paying $0.03/day thinks they have a cost problem.
Batch jobs and data pipelines can invert this failure mode. They are often piloted against historical data at full or near-full volume, which means the pilot cost may be representative, but teams forget that production runs more frequently than the pilot did. The math changes. The need to model it before deployment does not.
Three factors consistently drive teams to underestimate production volume.
Integration multipliers
A workflow that “makes one API call” usually makes three to five when you trace the full execution path: input pre-processing, the primary inference call, output parsing, a validation pass, and error handling with retry logic. Each of these may be a billable call.
A workflow modeled as “one call per user action” is frequently running three to five calls per user action in production. In more complex workflows with fallback paths to higher-capability models on failure, a single user-triggered event can cascade to significantly more calls than the happy path suggests.
Operational frequency drift
Pilots run when someone remembers to trigger them. Production workflows run on schedule, triggered by business events, and invoked by systems that don't forget.
A document processing pipeline that ran 20 times during a two-week pilot frequently runs 2,000 times in its first month of production — not because of unusual load, but because it is now integrated into the business process it was designed for.
Growth
Adoption of successful internal AI tools tends to compound. The cost model built at launch is already outdated before the first billing cycle closes.
The formula
Daily cost =
((input tokens per call × input price per 1M tokens)
+ (output tokens per call × output price per 1M tokens))
÷ 1,000,000
× calls per day
Input and output tokens are typically priced differently. Output tokens are often materially more expensive than input tokens — frequently several times higher, depending on provider and model tier.
Model the two token streams separately. Conflating them consistently understates the cost.
Run this estimate three times before deployment.
P50: expected production volume
Base this on the business process the workflow replaces or augments — not pilot data. If the workflow processes invoices and the business processes 800 invoices per day, that is the baseline.
P95: peak load
Month-end close, campaign launches, high-traffic events. What does load look like on the highest-volume day of the month?
Six-month growth
If adoption doubles, does the cost model still produce a positive ROI? If that conversation requires the business to revisit the investment case, it is easier before deployment than after the invoice.
Jake Cooper, the founder of Railway, described the pattern directly after shifting his 35-person team entirely to AI-assisted development — the company's Anthropic spend reached approximately $200,000 per month. 3
His observation on teams building AI applications: companies get “CFO bottlenecked before anything technical.” The spend outpaces the justification before anyone builds a cost model at production volume.
The pre-deployment cost audit
Five steps. The full exercise takes under an hour for a single workflow.
1. Draw the call map
List every inference call the workflow makes across the full execution path — including pre-processing, validation, error handling, and retry paths.
Do not model the happy path only. Observability tools that trace API calls, such as LangSmith, Helios, or vendor-native logging, can surface calls that are not obvious from reading the application code. The goal is the actual call graph, not the intended one.
2. Classify each call
For each call in the map, determine whether it is pattern recognition, context assembly, reasoning/synthesis, or agentic work. This determines which model tier is a candidate for production.
3. Match model to task — then validate
For every call classified as pattern recognition or context assembly, identify a candidate lower-cost model. Then run it against a representative sample of real inputs with defined acceptance thresholds before committing it to the architecture.
The pre-deployment cost audit and the model eval are the same project.
4. Build the volume estimate
Use the business process the workflow is joining, not the pilot data. Apply the integration multiplier — actual calls per workflow execution, not the happy-path count. Run P50 and P95. Model six-month growth.
5. Produce a monthly cost number at the right model tier
This is the number the business needs before approving deployment — and the number that answers the first question from Part 1 of this series: what does this workflow cost per run, at actual production volume? 6
If that number is materially different from what the business approved at pilot greenlight, that conversation belongs before the deployment decision.
The cost conversation this enables
The inference subsidy that characterized early AI development appears to be narrowing. 4 As frontier model companies move toward pricing that better reflects compute cost, workflows built on unnecessarily expensive models will become harder to defend — and harder to unwind.
The $8-in-compute-per-$1-of-subscription-revenue dynamic has to resolve somewhere. 5
The engineering team that runs this audit before deployment gives the business a defensible cost number, selects the right model for the task, and owns the cost model before someone else is handed the invoice and asked to explain it.
The model selection decision happens once, in development, under deadline pressure. The invoice arrives quarterly, with no deadline attached.
Getting ahead of that gap is a 45-minute exercise — and the only moment it is not a difficult conversation is before the architecture is committed.
Endnotes
- “Your Claude API bill is higher than your revenue: Why simple Python tasks are blowing up AI costs” — CIO , 2026-05-21. https://www.cio.com/article/4175244/your-claude-api-bill-is-higher-than-your-revenue-why-simple-python-tasks-are-blowing-up-ai-costs.html
- “No APIs, No AI: Organizing Software Engineering for Today's AI Reality” — Gartner ThinkCast / Manjunath Bhat, March 2026.
- “Railway: The Agent-Native Cloud — Jake Cooper” — Latent Space , 2026-05-21.
- “The Debate Over Anthropic's New Product: Price or Existential Dread?” — AI Daily Brief / NLW, 2026-03-10.
- “Every AI Subscription Is a Ticking Time Bomb for Enterprise” — State of Brand , May 2026.
- “AI Costs Like Labor. Your Budget Doesn't Know That Yet.” — Part 1 of this series.
