TLDR: Deloitte Australia has issued a partial refund of over AU$97,000 to the Australian Department of Employment and Workplace Relations after a AU$440,000 report, which utilized generative AI, was found to contain numerous errors, including fabricated court quotes and non-existent academic references. The firm admitted to using Azure OpenAI GPT-4o in the report’s creation, highlighting concerns about AI ‘hallucinations’ in professional services.
Deloitte Australia has agreed to a partial refund of over AU$97,000 (approximately US$63,000) to the Australian Department of Employment and Workplace Relations (DEWR) following the discovery of significant errors in a AU$440,000 (US$290,000) report. The report, commissioned in December 2024, was intended to provide an independent assurance review of the Targeted Compliance Framework (TCF) and its IT system, which manages welfare and benefits payments.
The original 237-page document, published in July, was found to be riddled with inaccuracies. These included a fabricated quote from a federal court judgment and references to nonexistent academic research papers. Dr. Christopher Rudge, a Sydney University researcher specializing in health and welfare law, was the first to publicly identify these mistakes, describing them as ‘hallucinations’—a term used when generative AI models produce false or inaccurate information. Rudge criticized the errors, stating, ‘That’s about misstating the law to the Australian government in a report that they rely on. So I thought it was important to stand up for diligence.’
Following the revelations, Deloitte reviewed the report and confirmed that ‘some footnotes and references were incorrect.’ A revised version was subsequently published, which included a new disclosure: ‘Deloitte’s methodology included the use of a generative artificial intelligence (AI) large language model (Azure OpenAI GPT – 4o) based tool chain licensed by DEWR and hosted on DEWR’s Azure tenancy.’ The updated report corrected more than a dozen references and footnotes, rewrote the reference list, and fixed typographical errors, though the DEWR stated that the core findings and recommendations remained unchanged.
The refund represents less than a quarter of the original payment, a point of contention for some critics. Labor Senator Deborah O’Neill sharply criticized Deloitte’s approach, remarking, ‘Deloitte has a human intelligence problem. This would be laughable if it wasn’t so lamentable. A partial refund looks like a partial apology for substandard work. Anyone looking to contract these firms should be asking exactly who is doing the work they are paying for.’
Also Read:
- Deloitte Unveils Major Enhancements to AI-Powered Delivery Platform, Ascend
- AI Pioneer Geoffrey Hinton Argues Job Displacement is a Societal Choice, Not Inherent AI Flaw
This incident is not isolated, reflecting a growing trend of generative AI systems fabricating information. Recent examples include a ‘fabricated’ interview by Michael Schumacher and Google having to correct false content about journalist Anabela Natário. In April 2023, an Australian mayor also pursued a defamation lawsuit against ChatGPT for falsely claiming he had been detained for bribery, highlighting the legal and ethical challenges posed by AI-generated content.
