Artificial intelligence costs are becoming a “material margin risk” for companies as rising token consumption and fluctuating vendor pricing complicate efforts to forecast the expense of scaling the technology, according to Gartner researchers.
Pressure to accelerate AI adoption without adequate governance and financial controls can expose companies to sharp cost increases, the researchers warned in a report last month.
“It’s really about making sure that unmonitored AI use and overly aggressive license rollouts don’t end up costing you millions of dollars as an organization,” Dennis Gannon, a vice president of research at Gartner and one of the report’s authors, said in an interview.
The emerging risk comes as CFOs also face pressure from boards and CEOs to deliver measurable gains from AI spending. However, 83% of CFOs say fewer than half of their organizations’ AI initiatives have produced such results, according to the report.
Separate research from KPMG underscores the challenge companies face as they try to control rising AI spending. In that firm’s second-quarter survey of senior leaders, 29% of respondents said their organizations struggle to understand and control AI operating costs as systems scale. Nearly half said their organizations have been forced to adjust AI deployment plans due to cost challenges.
Cost control gaps
The KPMG report also highlights gaps in organizations’ AI cost controls. Just over half of surveyed senior leaders said their organizations incorporate cost reviews into AI approval processes or use dedicated cost-monitoring dashboards. Fewer have established token budgets or architecture standards designed to improve efficiency.
Gartner warned that a “single unmonitored employee prompt library or unrestricted license rollout can risk spiking millions of dollars in additional expense.”
The material margin risk flagged by Gartner is largely prospective, Gannon said. “The ultimate trajectory of the AI cost curve is still an unknown,” he said, adding that commoditization and competitive pressure could push prices down, while market dynamics, energy costs and other factors could put upward pressure on them.
Gartner recommends that CFOs address AI costs as an architecture issue rather than a budget problem.
“Token consumption is largely determined by architectural choices made during system design,” the report said. “Context management, workflow design, orchestration patterns, retrieval strategies and agent interactions ultimately determine cost behavior.”
Gannon said finance leaders should work with IT and business teams on decisions involving model selection, AI agent orchestration, the information AI systems process and runtime constraints.
He also recommended monitoring token usage across individuals, teams, business units and applications, with clear accountability for spending. He pointed to operational controls such as routing requests to less expensive models when appropriate, caching prompts and limiting runtime as ways to help constrain costs.
Vendor management is another key area, according to Gannon. This includes steps such as seeking transparent pricing, scrutinizing contracts for hidden charges and developing the expertise to negotiate terms as AI usage grows, he said.
