The Token Paradox
Token prices have collapsed from roughly $20 to $0.07 per million tokens for the same capability, according to the Stanford HAI 2025 AI Index cited in McKinsey’s July 2026 report. Logic suggested that as the cost of the raw material dropped, the cost of building agentic AI would follow suit. Instead, LLM expenditures tripled over a 12-month period ending in 2025, as noted by Menlo Ventures. The economic model has shifted away from simple, token-centric inference toward a complex, process-centric reality.
Budgetary Friction
The McKinsey Enterprise AI FinOps Survey from May 2026 found that 93% of enterprises are currently exceeding their AI budgets. A Beri.net analysis of 127 enterprise implementations confirms this, showing that 73% went over budget, with some projects exceeding initial estimates by 2.4 times. This gap often translates into millions of dollars in unanticipated costs.
The Cost of Refinement
Why is this happening? Unlike a simple chatbot prompt, agentic tasks can consume roughly 1,000 times the tokens of a standard query. Furthermore, the McKinsey QuantumBlack report from July 2026 highlights that 60% of total agentic AI spend is consumed by response refinement—the iterative process of checking, correcting, and improving outputs—rather than the initial inference call.
Consider a banking case study from Beri.net. In this scenario, the actual token cost accounted for only 22% of the total per-loan AI cost. The remaining 78% was swallowed by tool calls, vector queries, human review, and compliance requirements. This breakdown illustrates the new reality: the model is the cheapest part of the equation.
Organizational Oversight
This financial pressure is already changing behavior. The McKinsey 2026 State of AI survey reports that 1 in 5 organizations have already been forced to constrain their AI use due to these operating costs. Meanwhile, the FinOps Foundation notes that 98% of practitioners are now managing AI spend, a massive jump from just 31% in 2024. Yet, there is a structural disconnect in oversight: only 8% of these FinOps teams report directly to the CFO, while 78% report to the CTO or CIO.
This reporting structure suggests that AI is still being treated as a technical challenge rather than a core financial one. If the current trajectory holds, the consequences will be stark. Gartner projected that over 40% of agentic AI projects will be canceled by the end of 2027, citing escalating costs, unclear business value, and inadequate risk controls.
The Path Forward
For enterprise leaders, the path forward requires a shift in perspective. The relevant question is no longer just about the cost of the model. It is whether the output an agent produces is worth the full cost of producing it, including the refinement cycles, the human supervision time, and the cost of correcting errors downstream. As the industry matures, the focus will likely move from the hype of what an agent can do to the cold, hard math of what it actually costs to keep it running.