Big Four accounting and consulting firm Deloitte’s Australian arm has come under public scrutiny after reports that it produced a document for the Australian government that was full of artificial intelligence-generated errors.
The incident involving one of the biggest names in the corporate finance world should serve as a wake-up call for anyone in the profession using AI tools without proper safeguards, experts say.
“This situation underscores a critical lesson for finance professionals: AI isn't a truth-teller; it’s a tool meant to provide answers that fit your questions,” Bryan Lapidus, FP&A Practice director for the Association for Financial Professionals, which provides training and credentials for people in the finance profession, said in an email.
Deloitte Australia’s report to the Australian Department of Employment and Workplace Relations was originally published on the department's website in July, according to an Associated Press article. A revised document was published earlier this month after revelations that the original version contained errors, including a fabricated quote from a federal court judgment and references to nonexistent academic research papers, AP reported.
Deloitte Australia agreed to partially refund the 440,000 Australian dollars ($290,000) paid by the Australian government for the document, which reviewed the information technology system used by the department to automate penalties in Australia’s welfare system.
“Deloitte conducted the independent assurance review and has confirmed some footnotes and references were incorrect,” a DEWR spokesperson said in an email, adding that “the substance” of the review has been retained. Deloitte has agreed to repay the final installment under its contract, the spokesperson said.
Deloitte Australia didn’t immediately respond to a request for comment.
Chris Rudge, a Sydney University researcher, has been credited with bringing the errors to light. He discovered that Deloitte’s report was “littered with citation errors,” prompting concern that it may have been generated by AI, according to an article published by the Australian Financial Review in August.
The event is a reminder for companies that AI is “smart but not infallible,” according to Nikki MacKenzie, an assistant professor in the Georgia Institute of Technology’s Scheller College of Business.
“We’re constantly hearing about how ‘intelligent’ AI has become, and that can lull people into trusting it too much,” she said in an email. “Whether consciously or not, we start to over-rely on it.”
AI tools such as generative AI chatbots can be susceptible to “hallucinations,” where they create outputs that are “nonsensical or altogether inaccurate,” according to an explainer published by ChatGPT creator OpenAI. Hallucinations can occur when an AI model is trained on a dataset comprising biased or unrepresentative data. AI tools can also be vulnerable to adversarial attack, where a bad actor manipulates output by subtly tweaking the input data, the article said.
Nearly six out of 10 employees admit to making mistakes in their work due to AI errors, according to a KPMG study released in April. In addition, about half use AI in the workplace without knowing whether it’s allowed and more than four in 10 are “knowingly using it improperly” at work, the study found.
The Deloitte event, while alarming, is unlikely to result in a pause in AI adoption, according to experts.
“I believe firms will see this as a normal cost of doing business,” MacKenzie said. “Just like how employees make mistakes, tools can too. The goal isn’t to avoid AI’s errors — it’s to make sure we’re smart enough to catch them as the ultimate decision-maker.”
The Deloitte incident is just the latest high-profile example of the risks posed by AI as it gets rapidly deployed across industries.
In January, Apple suspended an AI feature designed to summarize news alerts after complaints that it generated false and inaccurate information. And in 2023, a New York federal judge sanctioned two New York lawyers who submitted a legal brief that included fictitious case citations generated by ChatGPT.
Jack Castonguay, an associate professor of accounting at Hofstra University, said Deloitte is the first major accounting firm he’s aware of that has produced a report with fictitious details generated by AI.
“It seems like it was only a matter of time,” he said in an email. “Candidly, I’m surprised it took this long for it to happen at one of the firms.”
The event underscores the need for organizations to build AI safeguards or internal controls to mitigate risks such as hallucinations, MacKenzie said.
“The responsibility still sits with the professional using it,” she said. “Accountants have to own the work, check the output, and apply their judgment rather than copy-and-paste whatever the system produces.”
