“There are things known and there are things unknown and in between are the doors of perception.” — Aldous Huxley

I’m Huxley Westemeier (26’) and welcome to “The Sift,” a weekly opinions column focused on the impacts and implications of new technologies.

______________________________________________________

If you have ever used ChatGPT to assist with a bibliography and assumed it got the sources right, congratulations: both you and the South African government have something in common.

On April 10, South Africa’s Department of Communications and Digital Technologies published its Draft National Artificial Intelligence Policy. The document was 86 pages with 67 academic citations (I promise I’m not making that number up), and proposed a National AI commission, an AI Regulatory Authority, an AI Ethics Board, an AI Insurance Superfund (whatever that does), and best of all, an AI Ombudsperson, a real title for a person who investigates complaints and disputes within an organization. The bill was approved by the South African Cabinet back in March and was supposed to help position South Africa as a continental leader in AI.

One small issue soon emerged. South Africa’s own News24 verified the sources and found that at least six of the citations referenced real journals (for example, the Journal of Ethics and Social Philosophy), but the specific articles they cited had never been published. According to “Mashable,” additional editors confirmed that the citations were nonexistent and the policy was quietly withdrawn on April 26.

How did that happen?

If you ask older variants of ChatGPT or other Large Language Models (LLM) how many r’s are in the word “strawberry,” there is a decent chance it will fail along the way (newer models have memorized the answers). It might seem counterintuitive that a chatbot can explain challenging scientific concepts or write code, yet still fail at basic logic puzzles or identify the locations of letters within words. But that’s all because of the architecture. LLMs are physically incapable of retrieving information. They simply predict the most plausible next word based on patterns in their training data, and they are extraordinarily capable at sounding correct and unreliable at verifying whether what they are saying is true. So, if you ask one to draft a policy document with citations, it might generate citations that look correct (real journal references, formatting, plausible author names), because it has seen countless examples of citations. But it lacks a mechanism to verify whether those papers actually exist. Current frontier models from OpenAI and Anthropic tend to use their web search capabilities to minimize hallucinations, but those guardrails are not perfect.

South Africa is far from being the only culprit. According to “TechNext24,” a Nature study found that 2.6% of all academic papers published in 2025 (over 110,000 papers) contained at least one AI-generated citation, a dramatic increase from 0.3% in 2024. Also, at NeurIPS2025 (a prominent global AI conference), at least 100 fake citations were found in already peer-reviewed work. Even an education reform report in Newfoundland that called for the ethical use of AI in schools contained 15 fabricated sources that were easily detected.

The political fallout happening this week in South Africa is entertaining to a point. According to the British publication “The Register,” Khusela Sangoni-Diko (the chair of South Africa’s communications committee) publicly told Minister Malatsi to remove the document, suggesting that any rewrites will happen “without using ChatGPT this time.” Malatsi later posted on X that the lapse “proves why vigilant human oversight over the use of artificial intelligence is critical,” which would be a more satisfying response if it weren’t the exact lesson the withdrawn policy was supposed to promote.

To recap: someone let an AI write the bibliography for a document about why you shouldn’t blindly trust AI, and then nobody proofread the generated bibliography.

What is the AI Ombudsperson (again, a fantastic title) supposed to do about that?