STAMFORD, Conn., Aug 17: AI inference costs per agentic workflow will increase more than fivefold through 2028 according to Gartner, Inc., a business and technology insights company.
As AIproducts evolve from assistive features to multistep execution, product leaders face a new margin challenge with falling model prices subsidizing more complex workflows and escalating total AI costs. As a result, inference cost management has become a top priority for product leaders.
“Product leaders cannot rely on more efficient token economics to rationalize AI costs,” said Will Sommer, Sr. Director Analyst. “Each successive generation of AI capability will necessitate more, and often more expensive, tokens. There is no reliable, economical one-size-fits-all model on the horizon. Producing competitive AI products will require developing and maintaining complex multimodel ecosystems.”
Gartner has identified three fundamental trends driving token economics:
- Foundational model cost economics are rapidly improving.
- Improved AI efficiency is unlocking the deployment of more powerful, and more expensive, models to enable higher-value, more sophisticated AI applications.
- More sophisticated AI workflows use far more tokens than simple chatbot interactions, driving higher overall inference costs.
These dynamics mean that tokens are becoming more cost-efficient, but not as quickly as AI capabilities and the costs associated with those capabilities are increasing. The rate of innovation is outpacing the cost curve.
This is the Inference Paradox, defined as better unit economics escalating the overall cost of AI without providing a clear pathway to commensurate and predictable value.
“The harsh economics of the Inference Paradox are exemplified by the differences between a simple chatbot and an AI agent,” said Sommer. “Where a simple chatbot must read and interpret a query and quickly respond with a probabilistically reasonable answer, an AI agent must constantly reason, negotiate, and question itself.”
All of these responsibilities add up. Compared to a basic chatbot interaction, routing a task to an agentic reasoning model increases provider inference costs by at least five times, and often much more as task complexity grows.
Ensuring ROI from advanced AI, like reasoning agents, demands exponentially higher returns relative to basic models, or highly optimized inference-tiering, routing and orchestration to calibrate complex tasks relative to more cost-efficient intelligence. Both of these outcomes are eminently possible but will require significant effort across complex workflows.
“Defaulting to generic autonomous intelligence will result in unbounded costs orders of magnitude higher than those of optimized product ecosystems,” said Sommer.
Additional Insights Available
Gartner clients can read more in The Inference Paradox: Inference Tiering Is Critical to Protect Margins.
Gartner is the World Authority on AI
Gartner is the indispensable partner to C-Level executives and technology providers as they implement AI strategies to achieve their mission-critical priorities. The independence and objectivity of Gartner insights provide clients with the confidence to make informed decisions and unlock the full potential of AI. Clients across the C-Level are using Gartner’s proprietary AskGartnerAI tool to determine how to leverage AI in their business. With more than 2,500 business and technology experts, 6,000 written insights, as well as more than 4,000 AI use cases and case studies, Gartner is the world authority on AI. More information can be found here.
Gartner IT Symposium/Xpo
Additional analysis on the economics of AI will be presented during Gartner IT Symposium/Xpo, the world’s most important conference for CIOs and other IT executives. Gartner analysts and attendees will explore how to become agents of change in their organizations and harness AI for successful digital transformation. Follow news and updates from the conferences on XandLinkedIn using #GartnerSYM, and on the Gartner Newsroom.
Upcoming dates and locations for Gartner IT Symposium/Xpo include:
September 14-16, 2026 | Gold Coast, Australia
October 19-22, 2026 | Orlando, FLNovember 4-6, 2026 | Yokohama, Japan
November 9-12, 2026 | Barcelona, Spain
November 16-18, 2026 | Kochi, India