VICTOR: a Dataset for Brazilian Legal Documents Classification

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 1449–1458 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 1449 VICTOR: a dataset for Brazilian legal documents classification Pedro H. Luz de Araujo, Teofilo E. de Campos, Fabricio A. Braz, Nilton C. da Silva CiC and FGA, University of Brasília (UnB), Brasília - DF, Brazil pedrohluzaraujo@gmail.com, {teodecampos, fabraz, niltoncs}@unb.br Abstract This paper describes VICTOR, a novel dataset built from Brazil’s Supreme Court digitalized legal documents, composed of more than 45 thousand appeals, which includes roughly 692 thousand documents—about 4.6 million pages. The dataset contains labeled text data and supports two types of tasks: document type classification; and theme assignment, a multilabel problem. We present baseline results using bag-of-words models, convolutional neural networks, recurrent neural networks and boosting algorithms. We also experiment using linear-chain Conditional Random Fields to leverage the sequential nature of the lawsuits, which we find to lead to improvements on document type classification. Finally we compare a theme classification approach where we use domain knowledge to filter out the less informative document pages to the default one where we use all pages. Contrary to the Court experts’ expectations, we find that using all available data is the better method. We make the dataset available in three versions of different sizes and contents to encourage explorations of better models and techniques. Keywords: text classification, legal domain, language resources 1. Introduction Brazil’s legal system suffers from an unreasonably large number of lawsuits (de Cássia Carvalho Lopes, 2017). To put matters into perspective, about 80 million lawsuits were awaiting judgement in 2017. That is almost one process for every three Brazilians. The period from 2009 to 2017 saw an increase of 19.4 million lawsuits (Fariello, 2018). In addition, the average processing time of lawsuits can reach more than seven years in some cases. The long waiting times impact Brazil’s legal certainty and represent greater budgetary requirements—Brazil spent R$ 90.7 bil- lion in 2017 to maintain the judiciary, approximately 22 billion dollars (Secretaria de Comunicação Social do Con- selho Nacional de Justiça, 2018). Our work aims to apply Natural Language Processing (NLP) and Machine Learning (ML) techniques to Brazil’s Supreme Court (Supremo Tribunal Federal or STF) cases to help overturn this scenario. The STF receives roughly 42 thousand cases each semester, taking 22 thousand hours for humans to sort through. That time could be better spent at more complex stages of the judicial work flow, for instance the ones requiring legal reasoning. Most of the cases reach the court as PDF files with raster scanned documents. Approximately 10% of these are un- structured, containing several unindexed documents rang- ing from petitions and orders to rulings. Therefore, as a first goal we explore and evaluate methods for automat- ically classifying document types. The documents origi- nate in different Brazilian courts and often contain visual noise (handwritten annotations, stamps, stains). So the main challenges here are the intra-class diversity and the quality of the scanned documents. …

Content truncated to 500 words