The Subject Annotations of the Danish Parliament Corpus (2009-2017) - Evaluated with Automatic Multi-label Classification

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 1428–1436 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 1428 The Subject Annotations of the Danish Parliament Corpus (2009-2017) - Evaluated with Automatic Multi-label Classification Costanza Navarretta and Dorte Haltrup Hansen University of Copenhagen Emil Holms Kanal 2, Copenhagen S costanza@hum.ku.dk, dorteh@hum.ku.dk Abstract This paper addresses the semi-automatic annotation of subjects, also called policy areas, in the Danish Parliament Corpus (2009-2017) v.2. Recently, the corpus has been made available through the CLARIN-DK repository, the Danish node of the European CLARIN infrastructure. The paper also contains an analysis of the subjects in the corpus, and a description of multi-label classification experiments act to verify the consistency of the subject annotation and the utility of the corpus for training classifiers on this type of data. The analysis of the corpus comprises an investigation of how often the parliament members addressed each subject and the relation between subjects and gender of the speaker. The classification experiments show that classifiers can determine the two co-occurring subjects of the speeches from the agenda titles with a performance similar to that of human annotators. Moreover, a multilayer perceptron achieved an F1-score of 0.68 on the same task when trained on bag of words vectors obtained from the speeches’ lemmas. This is an improvement of more than 0.6 with respect to the baseline, a majority classifier that accounts for the frequency of the classes. The result is promising given the high number of subject combinations (186) and the skewness of the data. Keywords: Parliamentary Records, Corpus Annotation, Multi-label Classification 1. Introduction The interest in analysing and automatically processing large amounts of political data has increased in the past decades, and the research has been made possible by the digital availability of political data. One type of po- litical material consists of political speeches from par- liaments. These have been the object of investigation in common initiatives, such as the ParlaClarin workshops (Fiˇser et al., 2018; Fiˇser et al., 2020), and large data col- lections, such as the EuroParl parallel corpus (Koehn, 2005) and the ParlaMint corpora (Erjavec et al., 2022). The classification of political discourse in general po- litical subjects, also known as policy areas or domains, has been addressed by researchers in especially polit- ical sciences for many decades since subject classifi- cations facilitate the analysis and comparison of how politicians from different political wings address spe- cific subjects nationally or internationally, and over time. In the present work, we follow this line of research. More specifically, we address the annotation of po- litical subjects in the speeches of the Danish Parlia- ment Corpus (2009-2017) v2, with subject annotations, which was released in CLARIN-DK in spring 2021 (Hansen and Navarretta, 2021)1. The paper describes how the political subjects were semi-automatically an- notated in the corpus using the meetings’ agenda titles. The paper also presents an analysis of the annotations and an evaluation of the consistency of the annotations …

Content truncated to 500 words