TLDR: Despite the widespread adoption of AI chatbots in South Africa, experts are cautioning users about the pervasive issue of ‘AI hallucinations’ – the fabrication of inaccurate information. While ChatGPT continues to be the most popular AI tool, users are advised to cross-reference AI-generated content with verified sources to avoid potential pitfalls.
Artificial intelligence chatbots, including OpenAI’s ChatGPT, have become an integral part of daily life for many South Africans, assisting with work, education, and entertainment. However, a significant concern has emerged regarding the accuracy of information provided by these generative AI tools, leading experts to issue warnings about their use.
According to a MyBroadband poll conducted in April 2025, OpenAI’s ChatGPT remains the dominant AI tool among tech-savvy South Africans, with approximately 42% of respondents indicating its use. Google’s Gemini followed in second place with about 12.7% of the votes, and xAI’s Grok secured third with 11.4%.
Despite their popularity, generative AI tools are prone to ‘hallucinations,’ a phenomenon where they produce completely inaccurate or fabricated information. Siphumelele Zondi, a Durban University of Technology lecturer and technology expert, highlighted this danger, explaining that AI can present non-existent information in a convincing manner, making it difficult for users to discern its falsehood. “You would believe that it exists because of the way it puts it together and how strongly it makes a case for that particular situation or that particular person or that information,” Zondi stated. He added, “It seems to be all across the board when it comes to these AI tools when we are looking for information. It’s something that they can’t fix just yet and something that they can’t get right.”
This issue is not exclusive to ChatGPT. Other prominent AI tools like Google’s Gemini and Apple Intelligence have also exhibited similar problems. Zondi emphasized that AI models are designed to always provide an answer, regardless of its veracity. “They always have an answer. Whether it’s right or wrong, they’ll give you an answer,” he noted.
The consequences of AI hallucinations can be severe. A notable international incident involved a Norwegian man who launched a defamation case against OpenAI after ChatGPT falsely claimed he had murdered his children. Closer to home, a South African law firm reportedly faced difficulties after using non-existent, likely AI-generated, case citations.
In light of these challenges, experts strongly advise users to exercise caution. Zondi recommended, “Move away and then go to what has been authenticated, to information that has been verified by various sources. because, with hallucinations, AI can give you just about any information that it puts together from the internet.” This guidance underscores the importance of cross-referencing AI-generated content with reliable sources to mitigate the risks associated with inaccurate information.
Also Read:
- The Unforeseen Costs of AI: Businesses Grapple with Millions in Error Rectification and Rising Expenses
- Student Appeals Against AI Plagiarism Detection Tools Upheld by UK Ombudsman
Recent events also include an incident in May 2025 where xAI’s Grok chatbot, integrated into Elon Musk’s X app, exhibited an ‘unauthorized modification’ that led it to repeatedly discuss South African racial politics, further highlighting the unpredictable nature of these advanced systems.
