A Generalized Approach to Protest Event Detection in German Local News

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 3883–3891 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 3883 A Generalized Approach to Protest Event Detection in German Local News Gregor Wiedemann1, Jan Matti Dollbaum2, Sebastian Haunss2, Priska Daphi3, Larissa Daria Meier3 Leibniz Institute for Media Research1, University of Bremen2, University of Bielefeld3 g.wiedemann@leibniz-hbi.de, dollbaum@uni-bremen.de, haunss@uni-bremen.de, priska.daphi@uni-bielefeld.de, larissa.meier@uni-bielefeld.de Abstract Protest events provide information about social and political conflicts, the state of social cohesion and democratic conflict management, as well as the state of civil society in general. Social scientists are therefore interested in the systematic observation of protest events. With this paper, we release the first German language resource of protest event related article excerpts published in local news outlets. We use this dataset to train and evaluate transformer-based text classifiers to automatically detect relevant newspaper articles. Our best approach reaches a binary F1-score of 93.3 %, which is a promising result for our goal to support political science research. However, in a second experiment, we show that our model does not generalize equally well when applied to data from time periods and localities other than our training sample. To make protest event detection more robust, we test two ways of alternative article preprocessing. First, we find that letting the classifier concentrate on sentences around protest keywords only slightly improves the performance for in-sample data. For out-of-sample data, in contrast, binary F1-scores improve up to +4 percentage points (pp). Second, against our initial intuition, masking of named entities during preprocessing does not improve the generalization of protest event detection models in terms of F1-scores. However, it leads to a significantly improved recall of the models. Keywords: protest event detection, protest event analysis, text classification, computational social science 1. Introduction Social scientists conduct protest event analysis (PEA) to learn about developments and trends of the forms, scale and hot topics of political protests to draw con- clusions about the state of social cohesion and civil society (Koopmans and Rucht, 2002). For this, large samples of articles from one newspaper are usually se- lected by a list of key terms, then read manually and judged by relevancy whether they actually contain a protest event. For relevant articles, the protest event in- formation such as the number of participants and topic is extracted. One problem is that keyword-based arti- cle selection produces a high number of false positives because indispensable keywords like “demonstrate” (in German: demonstrieren) are ambiguous and can mean either “participate in a protest” or “to show how some- thing works”. Weeding out all false positives by read- ing all articles is very time-consuming. In our project “Protests and Social Cohesion: Comparing Local Con- flict Dynamics”, we, therefore, strive to conduct PEA in a mostly automatic pipeline of subsequent natural language processing tasks. This paper describes our efforts for the first task, protest event detection. Our general research goal is to derive a near-complete picture of public protest actions carried out between …

Content truncated to 500 words