TLDR: The Bombay High Court has invalidated an income tax assessment order against KMG Wires Pvt Ltd, criticizing the assessing officer for using ‘non-existent’ and ‘AI-generated case laws’ to justify a significant increase in the company’s taxable income. The court underscored the critical need for quasi-judicial authorities to verify AI-generated information, remanding the case for a fresh hearing.
In a landmark ruling that highlights the growing complexities of integrating artificial intelligence into legal and administrative processes, the Bombay High Court has quashed an income tax assessment order, flagging the assessing officer’s reliance on ‘non-existent’ and ‘AI-generated case laws’. The decision, delivered by a bench of Justices BP Colabawalla and Amit Jamsandekar earlier this month, underscores a crucial message for quasi-judicial authorities: the imperative to cross-verify information sourced from AI systems.
The case involved KMG Wires Pvt Ltd, whose income tax assessment for the current year was significantly altered by the National Faceless Assessment Centre (NFAC). The NFAC had assessed the company’s total income at Rs 27.91 crore, a substantial increase from the Rs 3.09 crore originally declared by the company. This revised assessment, along with a subsequent demand notice issued in March this year, was challenged by KMG Wires Pvt Ltd.
The High Court’s scrutiny revealed that the assessing officer, in calculating the peak balance and justifying additions to the company’s income, had cited legal precedents that were found to be both ‘non-existent’ and ‘AI-generated’. The court observed, ‘In this era of artificial intelligence (AI), one tends to place much reliance on the results thrown open by the system.’ However, it firmly stated that when exercising quasi-judicial functions, ‘such results (thrown open by AI) are not to be blindly relied upon, but the same should be duly cross-verified before using them.’
Furthermore, the court noted that the assessment order was passed in breach of the principles of natural justice, as it failed to adequately consider the reply filed by the company in response to the notice. This procedural lapse, combined with the reliance on unverified AI-generated legal material, led the court to invalidate the entire assessment order and the accompanying demand notice.
Also Read:
- India Mandates Clear Labeling for AI-Generated Content to Combat Digital Misinformation
- Navigating AI Chatbots: Essential Guidelines for Responsible Use and Critical Avoidances
The Bombay High Court has since remanded the issue back to the assessing officer for a fresh hearing, instructing that the process be conducted in accordance with established legal principles and due diligence. This ruling serves as a significant precedent, emphasizing the need for caution and rigorous verification when incorporating AI tools into critical decision-making processes, particularly within the legal and regulatory frameworks.
