NLP-Titan at SemEval-2023 Task 6: Identification of Rhetorical Roles Using Sequential Sentence Classification

Proceedings of the The 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 1365–1370 July 13-14, 2023 ©2023 Association for Computational Linguistics NLP-Titan at SemEval-2023 Task 6: Identification of Rhetorical Roles Using Sequential Sentence Classification Harsh Kataria Cisco Systems hkataria99@gmail.com Ambuje Gupta Uniphore Systems ambujegupta99@gmail.com Abstract The analysis of legal cases poses a consider- able challenge for researchers, practitioners, and academicians due to the lengthy and in- tricate nature of these documents. Develop- ing countries such as India are experiencing a significant increase in the number of pending legal cases, which are often unstructured and difficult to process using conventional methods. To address this issue, the authors have imple- mented a sequential sentence classification pro- cess, which categorizes legal documents into 13 segments, known as Rhetorical Roles. This approach enables the extraction of valuable in- sights from the various classes of the structured document. The performance of this approach was evaluated using the F1 score, which mea- sures the model’s precision and recall. The authors’ approach achieved an F1 score of 0.83, which surpasses the baseline score of 0.79 es- tablished by the task organizers. The authors have combined sequential sentence classifica- tion and the SetFit method in a hierarchical manner by combining similar classes to achieve this score. 1 Introduction Legal cases are often complex, lengthy and intri- cate, posing significant challenges for researchers, practitioners, and academicians alike. These chal- lenges become even more pronounced in devel- oping countries like India, where the number of legal cases is growing exponentially. The National Judicial Data Grid of India (NJDG, 2020) shows more than 43 million pending legal cases. These result in legal documents that are unstructured and highly complex, making them difficult to process efficiently. This requires modern solutions like new-age technology and AI advancements to help process this vast backlog of pending legal cases efficiently. Legal documents differ considerably from the text on which pre-trained Natural Language Pro- cessing (NLP) models are trained. Legal docu- ments are typically large, unstructured, and include legal jargon, as well as numerous mistakes. Ad- ditionally, different categories of legal cases are fundamentally different from one another. To over- come this challenge, NLP models trained on accu- rate well-annotated corpus are necessary. However, creating such a corpus itself is challenging, result- ing in slow growth in the Legal NLP domain. To efficiently process long legal documents, this paper proposes a method for rhetorical role classifi- cation using NLP techniques to segment legal texts from legal documents into semantically coherent classes. The proposed method is a text segmenting task which involves classifying parts or sentences of a document. The task uses a corpus of anno- tated legal documents, which have the spans of the document annotated with the appropriate class or rhetorical role. Sample rhetorical roles have been depicted in figure 1. A method to merge similar classes at the base level was implemented to aide the actual task. The sequence sentence classifi- cation was carried out using sequential sentence classifier and SetFit method is used …

Content truncated to 500 words