TeamUnibo at SemEval-2023 Task 6: A transformer based approach to Rhetorical Roles prediction and NER in Legal Texts

Proceedings of the The 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 275–284 July 13-14, 2023 ©2023 Association for Computational Linguistics TeamUnibo at SemEval-2023 Task 6: A transformer based approach to Rhetorical Roles prediction and NER in Legal Texts Yuri Noviello and Enrico Pallotta and Flavio Pinzarrone and Giuseppe Tanzi University of Bologna {yuri.noviello, enrico.pallotta, flavio.pinzarrone, giuseppe.tanzi} @studio.unibo.it Abstract This study aims to tackle some challenges posed by legal texts in the field of NLP. The LegalEval challenge (Modi et al., 2023) pro- poses three tasks, based on Indial Legal doc- uments: Rhetorical Roles Prediction, Legal Named Entity Recognition, and Court Judge- ment Prediction with Explanation. Our work focuses on the first two tasks. For the first task we present a context-aware approach to enhance sentence information. With the help of this approach, the classification model utilizing InLegalBert as a transformer achieved 81.12% Micro-F1. For the second task we present a NER approach to extract and classify entities like names of petitioner, respondent, court or statute of a given document. The model utiliz- ing XLNet as transformer and a dependency parser on top achieved 87.43% Macro-F1. 1 Introduction The exponential growth of pending legal cases in highly populated countries, such as India, has high- lighted the need for automation in the judicial pro- cess. Although a full automation of the legal do- main may not be feasible, intermediate tasks can be automated to assist legal practitioners and im- prove efficiency. Nevertheless, the unique nature of legal texts presents challenges for direct applica- tion of NLP models and techniques developed for common texts, thus requiring the development of specialized NLP approaches specifically tailored to the legal domain. The objective of the first task is to segment a given legal document by predict- ing the rhetorical role label for each sentence such as a preamble, fact, ratio, arguments, etc. These are referred to as Rhetorical Roles (RR). This segmentation is a fundamental building block for many legal AI applications like judgment summa- rizing, judgment outcome prediction, precedent search, etc. Legal documents are typically long (tens of pages), unstructured, noisy (e.g., grammat- ical and spelling mistakes due to manual typing in courts), and use different lexicons (legal jar- gon), which makes end-to-end transformer-based pre-trained models for sentence classification in- effective. In this paper we propose an approach based on the usage of transformer-based models pre-trained on legal text and fine-tuned on a Rhetor- ical Roles dataset in a context aware way, to make them able to accurately segment even long doc- uments. We first experimented with simple sin- gle sentence classification transformers (both pre- trained on general texts and legal texts) and then, leveraging on the fact that in a document nearby sentences classes tend to influence each other, we experimented enriching the sentence representa- tions with information coming from the context. First attaching 2 BiLSTM layers to the transformer embedding model and then simply extending the input tokens of the trasformer by adding the tokens from a context window around the sentence. …

Content truncated to 500 words