LTRC at SemEval-2023 Task 6: Experiments with Ensemble Embeddings

Proceedings of the The 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 841–846 July 13-14, 2023 ©2023 Association for Computational Linguistics LTRC at SemEval-2023 Task 6: Experiments with Ensemble Embeddings Pavan Baswani, Hiranmai Sri Adibhatla, Manish Shrivastava Language Technologies Research Center, KCIS IIIT Hyderabad, India pavan.baswani@research.iiit.ac.in, hiranmai.sri@research.iiit.ac.in, m.shrivastava@iiit.ac.in Abstract In this paper, we present our team’s involve- ment in SemEval-2023 Task 6: LegalEval: Un- derstanding Legal Texts. The task comprised three subtasks, and we focus on subtask A: Rhetorical Roles prediction. Our approach included experimenting with pre-trained em- beddings and refining them with statistical and neural classifiers. We provide a thorough ex- amination of our experiments, solutions, and analysis, culminating in our best-performing model and current progress. We achieved a mi- cro F1 score of 0.6133 on the test data using fine-tuned LegalBERT embeddings. 1 Introduction Rhetorical role labeling is a critical aspect of an- alyzing and interpreting legal documents, partic- ularly in the field of judgment. The labeling of rhetorical roles involves identifying the semantic purpose of each sentence and categorizing the dif- ferent roles played by each part of a judgment, including the parties involved, the arguments pre- sented, and the decision made. Identifying the different rhetorical roles of sentences in a legal case document can enhance the document’s com- prehension (Malik et al., 2021b) and aid in various downstream tasks, such as classification (Chalkidis et al., 2021), text summarization (Jain et al., 2021), case law analysis, and entity extraction (Skylaki et al., 2021; Chalkidis and Androutsopoulos, 2017; Chalkidis et al., 2017). Despite its potential benefits, the lack of struc- ture, technical jargon, multiple themes, and high specificity makes it difficult even for legal experts to identify the rhetorical roles effectively. There- fore, the task of rhetorical labeling in legal docu- ments is a taxing and challenging one that requires advanced NLP techniques. By understanding the various rhetorical roles at play, legal profession- als can more effectively analyze and interpret the content of legal documents, and ultimately, make informed legal decisions (Chalkidis et al., 2019). This process can help lawyers to understand the implications of the judgment, and ensure that their clients’ interests are protected in the legal system. While it may be challenging to automate the en- tire judicial process, automating intermediate tasks can be valuable in assisting legal practitioners and accelerating the system. Nevertheless, legal docu- ments possess unique characteristics, and the ap- plication of existing NLP models and techniques can be difficult due to the difference between legal texts and commonly occurring texts used to train such models. Consequently, the development of legal domain-specific techniques is essential to en- hance the application of NLP in the legal domain. As a step in that direction, Task 6 (LegalEval: Un- derstanding Legal Texts) of SemEval (Modi et al., 2023) proposed 3 shared subtasks which will act as building blocks in developing legal AI applications. In this paper, we describe our efforts on Sub-Task A: Rhetorical roles (RR) prediction1. 2 Related Work In recent years, there has been …

Content truncated to 500 words