Zero Tolerance for AI “Hallucinated” Case Law: Supreme Court Warns Judiciary

The Supreme Court set aside an NCLT insolvency order after finding it relied on fake, non-existent and AI-generated legal precedents , stressing a zero-tolerance approach to such errors in judicial proceedings.

A bench of Justices PS Narasimha and Alok Aradhe said courts and lawyers must not cite or rely on unverified AI-generated material , warning that even a small inclusion of such content can vitiate judicial decisions. The Court said advocates using such citations without verification may face professional misconduct , while judicial reliance on them is a serious lapse.

To highlight the danger, the bench compared the impact of such “hallucinated” legal content to the Bhopal Gas Tragedy’s methyl isocyanate leak , calling it “invisible, insidious and catastrophic,” underscoring how unnoticed errors in AI-generated content can later cause severe damage to the justice system.

The case involved the Essel Infraprojects insolvency matter, where fabricated Supreme Court judgments were cited in the NCLT’s reasoning. The Court held that even if the source of error was unclear, the decision cannot stand if based on false precedents .

The bench clarified it is not against AI use in law but emphasized human verification and control in adjudication . It also directed the Bar Council of India to frame guidelines and disciplinary measures to prevent such occurrences.