Subject: GS 2: Polity & Governance
Context: The Supreme Court has held that relying on AI-generated hallucinated (non-existent) judicial precedents is “catastrophic” to the judicial process.
- It set aside an NCLT order that had cited fictitious case laws generated by AI.
- The Court also directed the Bar Council of India (BCI) to formulate guidelines for the responsible use of Artificial Intelligence (AI) in legal practice
UPSC Online Preparation
What are AI Hallucinations?
- AI hallucinations occur when Generative AI models produce fabricated, non-existent or factually incorrect information while presenting it with high confidence, making it appear authentic.
- Legal Context: In judicial proceedings, AI may generate imaginary case laws, incorrect legal citations, misquoted judgments or non-existent statutory provisions, which can seriously mislead courts if not independently verified.
- Growing Global Concern: Several courts across jurisdictions, including the United States, have reported instances where lawyers relied on ChatGPT-generated fictitious precedents, prompting judicial sanctions and renewed emphasis on human verification.
Benefits of AI in the Judiciary
- Improving Judicial Efficiency: AI can significantly reduce the time spent on legal research, document review, translation, case summarisation and case listing, thereby helping address judicial backlog.
- Enhancing Access to Justice: AI-powered legal assistants can simplify legal information, improve multilingual access and reduce litigation costs for citizens.
- Administrative Modernisation: Intelligent automation can streamline registry functions, scheduling, record management and court administration.
- Supporting Digital Courts: AI complements initiatives such as e-Courts Mission Mode Project, Virtual Courts and SUVAS (Supreme Court Vidhik Anuvaad Software) by improving digital justice delivery.
Major Concerns Regarding AI in the Judiciary
- AI Hallucinations: Fabricated precedents, incorrect citations and inaccurate legal analysis can directly affect judicial outcomes and compromise justice.
Draft Supreme Court AI Regulations, 2026:
- Assistive—not Adjudicatory: AI may be used for legal research, translation, summarisation, case management and administrative support, but cannot replace judicial decision-making.
- No AI-Based Judicial Outcomes: AI systems cannot determine convictions, sentencing, grant or denial of bail, witness credibility, or any judicial outcome involving application of mind.
- Mandatory Disclosure: Lawyers must disclose whenever AI tools have substantially assisted in preparing pleadings, submissions or legal documents.
- Human Accountability: Judges remain fully accountable for every judicial order irrespective of AI-assisted drafting or research.
|
- Algorithmic Bias: AI models trained on biased datasets may reinforce existing social, economic or legal biases, affecting fairness and equality before law.
- Opacity of AI Systems: Many advanced AI models operate as “black-box algorithms”, making it difficult to explain how conclusions are reached.
- Privacy and Confidentiality: Processing sensitive judicial records through AI raises concerns regarding data security, confidentiality and informational privacy.
- Over-Reliance on Technology: Excessive dependence on AI risks weakening lawyers’ and judges’ analytical reasoning, legal research skills and independent application of mind.
- Accountability Gap: Determining responsibility becomes difficult when erroneous AI-generated outputs influence judicial proceedings.
UPSC Online Classes
Recent Developments in AI and Judiciary
- Draft Supreme Court AI Regulations, 2026: The Supreme Court recently released draft regulations proposing a human-in-the-loop approach, permitting AI only as an assistive technology.
- Justice B.R. Gavai’s Observations (2026): The Chief Justice-designate emphasised that AI can supplement judicial functions but can never substitute the human mind, constitutional values and judicial conscience.
- Global Experience: Courts in the United States, Canada and Australia have issued advisories cautioning lawyers against relying upon AI-generated legal research without independent verification after multiple instances of fabricated citations.
Way Forward
- Develop Comprehensive AI Governance: Establish clear legal and ethical standards governing AI deployment across the judicial ecosystem.
- Mandatory Verification Protocols: Institutionalise multi-layer verification of AI-generated legal research before it is cited before any judicial forum.
- Capacity Building: Regularly train judges, lawyers and court officials on AI capabilities, limitations and ethical risks.
- Promote Explainable AI: Encourage adoption of Explainable AI (XAI) systems whose reasoning can be audited and independently verified.
- Strengthen Data Governance: Develop secure AI systems trained on authenticated Indian legal databases while ensuring compliance with the Digital Personal Data Protection Act, 2023.
- International Best Practices: Draw upon emerging AI governance frameworks such as the Council of Europe Framework Convention on AI, the EU AI Act and the UN Global Digital Compact while adapting them to India’s constitutional framework.
Click to Know UPSC OnlyIAS Coaching Centres
Conclusion
Artificial Intelligence has enormous potential to improve judicial efficiency, legal research and access to justice. However, the legitimacy of the judicial process ultimately rests upon human reasoning, constitutional morality, judicial independence and the Rule of Law. The Supreme Court’s judgment reinforces that while AI may assist courts, the responsibility for delivering justice can never be delegated to machines.
3 Jul 2026
