Improved Opinion Role Labelling in Parliamentary Debates

Improved Opinion Role Labelling in Parliamentary Debates Laura Bamberg University of Mannheim Ines Rehbein University of Mannheim Simone Paolo Ponzetto University of Mannheim {rehbein,ponzetto}@uni-mannheim.de Abstract This paper presents a model for German Opin- ion Role Labelling (ORL), using the data from the IGGSA-STEPS 2014 and 2016 shared tasks. We frame the problem as a token classi- fication task and employ a simple transformer- based model that achieves new state-of-the- art results on the data. Then we investigate whether we can further improve our model by transferring knowledge from a related task, i.e., Semantic Role Labelling. Our results show that, despite the small size of our data, this transfer learning step yields further im- provements for ORL, mostly regarding recall for target prediction. Finally, we present an er- ror analysis, showing where knowledge trans- fer from SRL can help and what is still difficult for German ORL. 1 Introduction The extraction of subjective expressions together with their opinion holders and targets is not only an important processing step for the analysis of argumentation mining but is also relevant for po- litical text analysis. For English, the seminal work of Stoyanov et al. (2004) and Wiebe et al. (2005) has provided resources for training and evaluation of opinion mining models for newswire. However, resources for other languages, domains and text types are still scarce. Previous work on German has focussed on the political domain where Ruppenhofer et al. (2014, 2016) have presented a corpus of Swiss-German parliamentary debates annotated with subjective expressions, their opinion holders (or sources) and targets (Figure 1). The data set has been used in two shared tasks.1 However, compared to the MPQA 2.0 corpus (Wiebe et al., 2005; Wilson, 2008) which includes more than 8,500 sentences, 1See the IGGSA-STEPS 2014 shared task: https://sites.google.com/site/ iggsasharedtask/task-1 and for 2016: https://iggsasharedtask2016.github.io. Figure 1: Screenshot of example annotations from the IGGSA-STEPS shared task data for the verb “bitten” (ask) and the noun “Zustimmung” (approval), visu- alised in Salto (Burchardt et al., 2006a). the data is rather small with less than 1,200 sen- tences. This is reflected in the low results for opinon holder and target extraction, where scores for the best systems from the 2016 shared task were in the range of 46% F1 (micro) for holders and 40% F1 for targets. Follow-up work by Wie- gand et al. (2019a) has improved the extraction of opinion holders by around 4 percentage points but failed to increase results for target extraction. The low results imply that, at this stage, the models are not yet good enough to be used in downstream applications. Since then, transformer-based models (Vaswani et al., 2017; Devlin et al., 2019) and transfer learn- ing approaches have brought huge improvements to the field of Natural Language Understanding (NLU) and are particularly well suited for task set- tings where only small data are available. There- fore, in our work we exploit the expressive power of transformers and transfer learning and present a simple transformer-based system for German opin- ion holder and …

Content truncated to 500 words