YNU-HPCC at SemEval-2023 Task 6: LEGAL-BERT Based Hierarchical BiLSTM with CRF for Rhetorical Roles Prediction

Proceedings of the The 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 2075–2081 July 13-14, 2023 ©2023 Association for Computational Linguistics YNU-HPCC at SemEval-2023 Task 6: LEGAL-BERT based Hierarchical BiLSTM with CRF for Rhetorical Roles Prediction Yu Chen, You Zhang, Jin Wang, and Xuejie Zhang School of Information Science and Engineering Yunnan University Kunming, China Contact: chenyv@mail.ynu.edu.cn, yzhang0202@ynu.edu.cn Abstract To understand a legal document for real-world applications, SemEval-2023 Task 6 proposes a shared Subtask A, rhetorical roles (RRs) pre- diction, which requires a system to automati- cally assign a RR label for each semantical seg- ment in a legal text. In this paper, we propose a LEGAL-BERT based hierarchical BiLSTM model with conditional random field (CRF) for RR prediction, which primarily consists of two parts: word-level and sentence-level encoders. The word-level encoder first adopts a legal- domain pre-trained language model, LEGAL- BERT, initially word-embedding words in each sentence in a document and a word-level BiL- STM further encoding such sentence represen- tation. The sentence-level encoder then uses an attentive pooling method for sentence em- bedding and a sentence-level BiLSTM for doc- ument modeling. Finally, a CRF is utilized to predict RRs for each sentence. The officially released results show that our method outper- formed the baseline systems. Our team won 7th rank out of 27 participants in Subtask A. 1 Introduction In populous countries, the number of pending legal cases has been rising exponentially. For instance, in India, according to the National Judicial Data Grid1, as of 04 July 2022, the Supreme Court of In- dia had approximately 6 million cases pending, and with the addition of other local courts, there will be far more than 6 million cases pending. It is urgent to require an automatic legal system, which helps practitioners extract accurate and valid information from legal documents, for efficient legal processing. To this end, SemEval-2023 proposes a shared Task 6 (Modi et al., 2023), LegalEval, for understand- ing legal texts, which mainly comprises of three subtasks including rhetorical roles (RRs) predic- tion, legal named entity recognition (L-NER), and 1https://njdg.ecourts.gov.in/ hcnjdgnew/?p=main/pend_dashboard court judgment prediction with explanation (CJPE), respectively. • Subtask A RR Prediction. Structuring unstruc- tured legal documents into semantically coher- ent units. • Subtask B L-NER. Identifying relevant enti- ties in a legal document. • Subtask C CJPE. Predicting the outcome of a case along with an explanation. Focusing on the RR prediction task, it requires a system to structure unstructured legal documents into semantical segments that are aligned with RR labels such as preamble, fact, ratio, etc. Such seg- mentations as fundamental building blocks are cru- cial for many legal artificial intelligence (AI) appli- cations, e.g., judgment summarizing, judgment out- come prediction, and precedent search. Due to the shared task having preprocessed legal documents by splitting them into several semantical sentences as coherent units, RR prediction in Subtask A could be regarded as a sentence-wise sequence labeling task that predicts a RR label for each sentence in a legal document. With the increasing growth of deep …

Content truncated to 500 words