TüReuth Legal at SemEval-2023 Task 6: Modelling Local and Global Structure of Judgements for Rhetorical Role Prediction

Proceedings of the The 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 262–269 July 13-14, 2023 ©2023 Association for Computational Linguistics TüReuth Legal at SemEval-2023 Task 6: Modelling Local and Global Structure of Judgements for Rhetorical Role Prediction Henrik Manegold University of Bayreuth Henrik.Manegold@uni-bayreuth.de Leander Girrbach University of Tübingen leander.girrbach@uni-tuebingen.de Abstract This paper describes our system for SemEval- 2023 Task 6: LegalEval: Understanding Le- gal Texts. We only participate in Sub-Task (A), Predicting Rhetorical Roles. Our final submission achieves 73.35 test set F1 score, ranking 17th of 27 participants. The proposed method combines global and local models of la- bel distributions and transitions between labels. Through our analyses, we show that especially modelling the temporal distribution of labels contributes positively to performance. 1 Introduction This paper describes our system for SemEval-2023 Task 6: LegalEval: Understanding Legal Texts. While the shared task hosts 3 sub-tasks, we only participate in sub-task (A), predicting Rhetorical Roles (RRs) from pre-segmented judgments. Refer to the overview paper (Modi et al., 2023) for a detailed description of the task and data. Briefly, the task is as follows: Given a judg- ment that is pre-segmented into paragraphs (often containing only 1 sentence) and an inventory of Rhetorical Roles, predict the appropriate RR for every paragraph. RRs structure the judgment into semantically coherent units and provide informa- tion on what is discussed in each unit. For example, the RR PREAMBLE denotes formal introductory statements of judgments. In total, there are 13 RRs in this task. The shared task provides 245 segmented and la- belled Indian judgements (in English) for training, 30 segmented and labelled judgments for devel- opment, and 50 segmented but unlabelled test set judgments for evaluation. Test set labels are not released by the organisers. The main challenge of this task is how to inte- grate document level information and paragraph level information. Since solving the task re- quires precise understanding of paragraph content and the context, fine-tuning pre-trained large lan- guage models (LMs) is the common starting point. However, LMs can not process long documents such as judgments on standard hardware at once. Therefore, we develop and evaluate a method to use document-level information in order to refine paragraph-level predictions from a fine-tuned LMs. In particular, we model the positions in a document where RRs usually occur, structural constraints that indicate whether some RRs may or may not be pre- dicted when other RRs have already been predicted earlier, and lexical constraints motivated by domain knowledge. Throughout this paper, we assume the following notation: R is the set of Rhetorical Roles, r ∈ R denotes some RR, a judgement J consists of a sequence of paragraphs s1, . . . , sT where T is the number of paragraphs in J. Paragraphs are indexed by t (we think of them as a time-series). 2 Related Work Closest to our method is previous work by the shared task organisers (Malik et al., 2021; Kalamkar et al., 2022). Both works propose to first encode each sentence of …

Content truncated to 500 words