New Delhi: In a landmark ruling, the Supreme Court of India has issued a stern warning against the unverified deployment of Artificial Intelligence in the legal system, declaring a "zero-tolerance" policy towards citing hallucinated legal precedents.
A bench comprising Justices P.S. Narasimha and Alok Aradhe made the critical observations while striking down corporate insolvency orders passed by the National Company Law Tribunal (NCLT) and its appellate body (NCLAT) in the Essel Infraprojects case. The apex court discovered that both lower forums had inadvertently relied on fake, non-existent judicial citations generated by AI software.
Comparing unregulated AI hallucinations to the catastrophic Bhopal gas leak, the bench remarked that the production and utilisation of fabricated legal material acts like the release of methyl isocyanate in the province of law and justice: "invisible, insidious, catastrophic."
The court clarified that it was not rejecting AI as a supportive tool for administration, but asserted that judicial determination must remain under absolute human control. In an unprecedented directive, the top court instructed the Bar Council of India (BCI) to set up an expert panel to investigate the issue, formulate ethical guidelines, and prescribe strict disciplinary actions for advocates who file unverified AI material.