Text Classification and Prediction in the Legal Domain

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 4717–4722 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 4717 Text Classification and Prediction in the Legal Domain Minh-Quoc Nghiem1,3,5, Paul Baylis3, Andr´e Freitas1,4, Sophia Ananiadou1,2,5 1 Department of Computer Science, The University of Manchester, United Kingdom 2 Alan Turing Institute, 3 Bott and Co Solicitors, 4 Idiap Research Institute, 5 National Centre for Text Mining {minh-quoc.nghiem, andre.freitas, sophia.ananiadou}@manchester.ac.uk, p.baylis@bottonline.co.uk Abstract We present a case study on the application of text classification and legal judgment prediction for flight compensation. We combine transformer-based classification models to classify responses from airlines and incorporate text data with other data types to predict a legal claim being successful. Our experimental evaluations show that our models achieve consistent and significant improvements over baselines and even outperformed human prediction when predicting a claim being successful. These models were integrated into an existing claim management system, providing substantial productivity gains for handling the case lifecycle, currently supporting several thousands of monthly processes. Keywords: text classification, legal judgment prediction, flight compensation 1. Introduction Natural language processing (NLP) technologies can play an important role in supporting legal processes. In certain circumstances, NLP technologies can be used to aid lawyers as they investigate or review facts or de- tails of a case. Additionally, it can also provide insight or automate the operational processes in legal work. There is increasing interest in the application of NLP technologies to the legal domain, and the field is slowly transforming as a result. Accessing legal services is often expensive and re- quires paying a high fee to traditional lawyers. Many law firms offer “no-win, no-fee” agreements to make access to legal services easier and more affordable, where you pay a percentage of any compensation you receive if the case is successful. It is common for per- sonal injury and flight compensation claims to be “no- win, no-fee”, and the amount recovered is usually rela- tively small. While trying to maintain a quality service, many law firms have to reject smaller cases without a significant margin. In order to operate profitably, law firms have to reduce operating costs as well as assess the likelihood of a claim being successful before enter- ing into a “no-win, no-fee” agreement with clients. In this paper, we present our methodology of develop- ing, deploying, and maintaining language processing technologies to classify airlines’ responses to the claim and predict litigation outcomes of flight compensation claims. We were able to embed the results into our existing case management system, which dramatically improved email and document management for the team of claims handlers. By combining a transformer- based model with human-in-the-loop, our methodolo- gies allow us to classify airlines’ responses and fore- cast litigation outcomes. We also provide the details of the framework to allow us to continuously improve the deployed systems through retraining the models with additional data which is systematically collected and labeled. Table 1 shows samples of airlines’ responses and the …

Content truncated to 500 words