Growing concerns about artificial intelligence safety and reliability could raise the stakes for CFOs evaluating potential investments in the technology, according to AI strategy experts.
While the debate is not expected to ease AI adoption pressures, experts say it could force deeper conversations about risk management as part of the investment decision-making process.
“I believe this is a turning point, but not necessarily a slowdown,” Jack McCullough, founder and president of the CFO Leadership Council, said in an email.
Dan Priest, U.S. chief AI officer at PwC, similarly said the debate could mark a shift toward greater AI preparedness and more emphasis on responsible use rather than a pullback in enterprise AI investment.
“Fear and trust have been headwinds for adoption and the risks are coming further into focus, but there are reasonable things that can be done to address safety concerns and still achieve the AI upside,” Priest said in an email. “The debate right now will likely accelerate these solutions.”
AI incidents put safety in focus
Recent incidents involving rogue AI models have triggered a broad safety debate and calls for action by Congress.
Anthropic disclosed in late July that it uncovered incidents in which some of its models escaped from a testing environment and gained unauthorized access to the systems of other organizations. The news came on the heels of OpenAI’s disclosure that AI models it developed were behind an “unprecedented cyber incident” affecting Hugging Face.
The debate further escalated this month when Jacob Coxon publicly resigned from his position as an Anthropic researcher, warning that the industry was moving too quickly toward self-improving AI systems.
Anthropic CEO Dario Amodei subsequently called for companies to slow the pace of frontier AI development, including through independent safety evaluators and greater coordination on safety standards. OpenAI CEO Sam Altman and xAI CEO Elon Musk publicly supported the proposal.
On Wednesday, OpenAI announced a new framework for tracking and publicly reporting model “misalignment” and disclosed six additional cases involving rogue AI behavior.
A bigger role for risk management
CFOs are already focused on questions about reliability, controls and accountability when evaluating AI investments. But an intensifying debate surrounding AI failures and unintended consequences could make those issues even more central to how companies evaluate the technology, McCullough said.
“This is no longer simply a technology decision,” he said. “It is a capital-allocation, risk-management and governance decision, all areas in which the CFO has an essential role.”
The central question is whether companies are creating value at a level of risk they understand and are prepared to accept, McCullough said.
“The CFO should not be the person saying no to AI,” he said. Instead, finance leaders should distinguish responsible adoption from investments driven primarily by hype or competitive pressure.
Putting a price on AI failures
For CFOs, the concern should not be limited to whether an AI system can improve productivity or deliver a return on investment, experts said. Failures could create financial exposure through erroneous payments or pricing decisions, business interruptions, regulatory penalties, litigation, data disclosure, intellectual property disputes and remediation costs.
The level of controls should depend on the use case. An AI tool used to help draft marketing materials presents a different risk profile from one used to make decisions involving treasury, financial reporting or cybersecurity, he said.
Priest said companies should fund the full cost of an AI initiative, including data, testing, monitoring, governance and workforce needs. Those costs should be incorporated into the value case from the beginning and weighed against the expected business outcome, he said.
As companies deploy AI more broadly, Priest said they also should consider an enterprise-wide control structure rather than evaluating each AI application in isolation.
“Investing in an intelligent control layer that monitors agents, measures quality and performance, and serves as an early warning system across major risks is also prudent,” he said.
Before approving a significant AI deployment, McCullough said finance leaders should consider questions such as: “What is the maximum credible loss if this system fails — and how quickly would we know?”
CFOs should also ask questions such as what data can the system access and whether the vendor may retain or use that data; what decisions or actions can the system take without human approval; and who is accountable if the system causes harm.
CFOs also should review insurance coverage and vendor contracts, he said, while testing, monitoring and governance should be included in an AI investment’s expected costs.
“A business case that includes labor savings but excludes the cost of making the technology safe is incomplete,” McCullough said.
Preparing for a moving target
Priest said companies should have a comprehensive AI preparedness plan covering responsible use, safety, risk management and incident response. That responsibility should extend beyond technology teams to business units adopting AI, he said, with appropriate requirements and accountability measures for third-party AI providers.
Potential regulatory changes should also be considered, Priest said, adding that companies are already seeing state-level requirements take effect, while federal policymakers weigh stepping in as well.
Priest advised CFOs to determine how proposed requirements could affect their companies and build reasonable compliance measures into their AI preparedness plans.
“For CFOs, the practical priority is to be prepared with a strong governance foundation that includes clear accountability, testing, monitoring and risk-based controls so the organization is ready to respond to different requirements,” Priest said.
