Nonet at SemEval-2023 Task 6: Methodologies for Legal Evaluation

Proceedings of the The 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 1293–1303 July 13-14, 2023 ©2023 Association for Computational Linguistics Nonet at SemEval-2023 Task 6: Methodologies for Legal Evaluation Shubham Kumar Nigam1 Aniket Deroy2 Noel Shallum3 Ayush Kumar Mishra1 Anup Roy1 Shubham Kumar Mishra1 Arnab Bhattacharya1 Saptarshi Ghosh2 Kripabandhu Ghosh4 1 IIT Kanpur, India 2 IIT Kharagpur, India 3 Symbiosis Law School Pune, India 4 IISER Kolkata, India {shubhamkumarnigam,roydanik18,noelshallum, saptarshi.ghosh,kripa.ghosh}@gmail.com {ayushkm20,puna20,skmishra20,arnabb}@iitk.ac.in Abstract This paper describes our submission to the SemEval-2023 for Task 6 on LegalEval: Un- derstanding Legal Texts. Our submission con- centrated on three subtasks: Legal Named En- tity Recognition (L-NER) for Task-B, Legal Judgment Prediction (LJP) for Task-C1, and Court Judgment Prediction with Explanation (CJPE) for Task-C2. We conducted various ex- periments on these subtasks and presented the results in detail, including data statistics and methodology. It is worth noting that legal tasks, such as those tackled in this research, have been gaining importance due to the increasing need to automate legal analysis and support. Our team obtained competitive rankings of 15th, 11th, and 1st in Task-B, Task-C1, and Task-C2, respectively, as reported on the leaderboard. 1 Introduction The SemEval Task-6 (Modi et al., 2023) aims to au- tomate several tasks to streamline the Indian legal process, which is often slow and delayed due to the country’s large population and a shortage of judi- cial resources. Additionally, people in India are not always fully aware of the country’s laws. To make the legal process more accessible to the general pub- lic, the SemEval task addresses crucial problems that are specific to the Indian judicial context. One task is the "Legal Named Entity Recognition (L- NER)" system, which identifies named entities in the legal text. Legal judgments contain intriguing entities like the names of the petitioner, respondent, judge, lawyer, date, organization, GPE, statute, pro- vision, precedent, case number, witness, and other persons, which are typically not recognized by con- ventional entity recognition systems. Therefore, developing systems that are tailored to the legal do- main is crucial. In task B, participants were tasked with identifying the legal entities present in legal judgments. A court judgment is divided into two parts: the preamble, which includes the names of the parties, the court, lawyers, and other details, and the decision (judgment), which follows the preamble. The organizers separately provided the preamble and judgment text datasets. Apart from the Legal NER system, the SemEval Task-6 also includes another important task C, that is crucial for automating the Indian legal process. The subtask, referred to as task C1, is the Legal Judgment Prediction (LJP) task. This task aims to determine whether the legal judgment favors the appellant or the defendant, which is modeled as a binary classification problem. Given a large num- ber of pending legal cases in India, automating the process of predicting legal judgments can signif- icantly reduce the burden on the judicial system. Moreover, the task of finding an explanation for the legal decision is equally important. For this, we …

Content truncated to 500 words