A New Dataset for Topic-Based Paragraph Classification in Genocide-Related Court Transcripts

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 4504–4512 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 4504 A New Dataset for Topic-Based Paragraph Classification in Genocide-Related Court Transcripts Miriam Schirmer†,‡, Udo Kruschwitz†, Gregor Donabauer† †University of Regensburg, Germany ‡TUM School of Social Science and Technology, Technical University of Munich, Germany {miriam.schirmer, udo.kruschwitz, gregor.donabauer}@ur.de Abstract Recent progress in natural language processing has been impressive in many different areas with transformer-based approaches setting new benchmarks for a wide range of applications. This development has also lowered the barriers for people outside the NLP community to tap into the tools and resources applied to a variety of domain-specific applications. The bottleneck however still remains the lack of annotated gold-standard collections as soon as one’s research or professional interest falls outside the scope of what is readily available. One such area is genocide-related research (also including the work of experts who have a professional interest in accessing, exploring and searching large-scale document collections on the topic, such as lawyers). We present GTC (Genocide Transcript Corpus), the first annotated corpus of genocide-related court transcripts which serves three purposes: (1) to provide a first reference corpus for the community, (2) to establish benchmark performances (using state-of-the-art transformer-based approaches) for the new classification task of paragraph identification of violence-related witness statements, (3) to explore first steps towards transfer learning within the domain. We consider our contribution to be addressing in particular this year’s hot topic on Language Technology for All. Keywords: Text classification, BERT, Professional Search, Genocide Studies, Language Technology for All 1. Introduction Information overload has led to a multitude of search applications of which Web search is just one out of many. Unlike search for leisure or personal interest there is a vast area of search contexts which are found in a work environment. Professional search falls into that scope, i.e. search over domain-specific document collections and often with search tasks that are recall- oriented rather than precision-focused (Kruschwitz and Hull, 2017; Verberne et al., 2019). Beyond applications where such search effort can directly be measured in financial terms (e.g. in patent search, e-discovery or the compilation of systematic reviews) there are many other fields where these costs are more implicit, e.g. in the area of genocide studies that rely on the analysis of vast quantities of different resources (Bachman, 2020; Hinton, 2012). Looking at the wider picture, searching large text cor- pora for specific thematic patterns can be very time- consuming and non-trivial, in particular for searchers who do not have a solid foundation in NLP or search technology. The huge amount of court transcripts of genocide tribunals presents a perfect example: the International Criminal Tribunal of the Former Yu- goslavia (ICTY) alone provides official transcripts for each of its cases online, leading up to approximately 2.5 million pages of transcripts in total (ICTY, 2016). Searching for specific content in a text corpus like this usually requires vast amounts of manual research capacity …

Content truncated to 500 words