4 questions CFOs should be asking about agentic AI initiatives

The business ramifications of agentic AI already extend far beyond the responsibilities of technology experts within the typical enterprise. Agentic AI, given its strategic importance coupled with its financial impacts, requires broader oversight by more than just technology leaders. Every leader in the C-suite, not just the CIO and/or CTO, but also the CEO—and especially the CFO—must have visibility into the execution of agentic AI initiatives. This requirement should be in place for most organizations, as 81% of organizations identified agentic AI as a strategic priority.^[1]

Agentic AI is designed to make independent decisions and take actions. This autonomy, however, often comes with a heavy price tag. Agentic workloads can increase inference transactions by 50 to 100 times that of traditional generative AI (GenAI) experiences. When using a chat interface with GenAI, a prompt is processed and a response is returned. With agentic AI, agents generate multiple responses as larger goals are broken into sub-tasks (such as planning, data retrieval and execution) and additional steps are generated to evaluate outputs and course correct when necessary. Token consumption and the associated costs for agentic workflows are often higher and less predictable as a result.

For the business, controlling the cost of agentic AI becomes a strategic necessity. Consider that:^[2]

42% of organizations expected to allocate $1m or more into AI agent initiatives over the following 12 months.
61% of organizations agreed that AI agent costs are a major barrier to adoption.

To best ensure profitability for your organization’s agentic AI initiatives, following are four questions to ask your AI leadership team when making new infrastructure purchases.

Are we using the latest technologies for our AI projects? With AI initiatives, the high cost of GPUs makes it critical to maximize their utilization (the percentage of time the GPU is in use rather than sitting idle waiting) and new innovation is making it much easier to keep GPUs fully utilized, reducing costs. For example, the recent integration of a technology called Dynamic Resource Allocation (DRA) into Kubernetes (a software platform for digital apps) enables developers to specify general hardware requirements and then the technology automatically finds an available resource that meets those requirements. This capability is important because, as recently as a year ago, developers of AI applications were required to specify the GPU for each app to use (e.g., app “A” gets GPU “1,” app “B” gets GPU “2,” etc.). This requirement often led developers to request new budget and new GPUs for each new AI project. The use of DRA should cut back on new GPU requests by automating the ability for AI apps to find and use existing idle GPUs, which should then significantly reduce AI costs.
Does this AI workload require a GPU? On the topic of GPUs, the assumption that every AI workload requires a GPU is both inaccurate and costly. According to our research, a larger percentage of organizations identified plans to invest in general-purpose CPUs (59%), AI-optimized CPUs (58%) and custom ASICs such as TPUs (56%) for their AI initiatives than NVIDIA GPUs (50%).^[3] While more powerful accelerators are important for heavyweight tasks, other processing options exist and can often provide better price-performance for inference operations that scale rapidly with agentic AI adoption.
How are we measuring ROI for this AI project? While the component cost differences, such as those between GPU and CPU resources, are critical to understand, focusing on individual chip prices alone oversimplifies the full cost of AI projects. For example, the complexity of the full infrastructure design, including its integrations, is a major determinant of the overall cost, one that public cloud-based solutions can help mitigate. Every AI investment should have a plan to measure and maximize ROI for the project, rather than simply minimizing the cost of components.
Can we reduce our procurement risk without overbuying? According to Omdia research, 68% of organizations agreed that current NAND flash supply chain concerns have impacted their storage strategy in 2026.^[4] The global demand for AI infrastructure has led to overbuying upfront to ensure supply. However, there are alternative options to pre-plan ahead of demand. For example, some Kubernetes platforms offer the ability to automatically provision pre-approved alternative hardware options, similar to providing a “plan b” or “plan c,” if the primary option is unavailable. This functionality provides greater flexibility in hardware choice and reduces the need to “overbuy” specific components ahead of time. For the technical specifics, here is a link to a blog that goes into greater detail.

Agentic AI is expected to be such a major expense that 45% of organizations plan to stand up completely new budgets for AI agent initiatives,^[5] and costs will rise quickly if left unchecked. Those with fiduciary responsibility for their business should apply extra scrutiny to these investments to best ensure that agentic AI delivers the value it promises.

References