ParlVote: A Corpus for Sentiment Analysis of Political Debates

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 5073–5078 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 5073 ParlVote: A Corpus for Sentiment Analysis of Political Debates Gavin Abercrombie and Riza Batista-Navarro Department of Computer Science, University of Manchester Kilburn Building, Manchester M13 9PL {gavin.abercrombie, riza.batista}@manchester.ac.uk Abstract Debate transcripts from the UK Parliament contain information about the positions taken by politicians towards important topics, but are difficult for people to process manually. While sentiment analysis of debate speeches could facilitate understanding of the speakers’ stated opinions, datasets currently available for this task are small when compared to the benchmark corpora in other domains. We present ParlVote, a new, larger corpus of parliamentary debate speeches for use in the evaluation of sentiment analysis systems for the political domain. We also perform a number of initial experiments on this dataset, testing a variety of approaches to the classification of sentiment polarity in debate speeches. These include a linear classifier as well as a neural network trained using a transformer word embedding model (BERT), and fine-tuned on the parliamentary speeches. We find that in many scenarios, a linear classifier trained on a bag-of-words text representation achieves the best results. However, with the largest dataset, the transformer-based model combined with a neural classifier provides the best performance. We suggest that further experimentation with classification models and observations of the debate content and structure are required, and that there remains much room for improvement in parliamentary sentiment analysis. Keywords: sentiment analysis, parliamentary debates, Hansard 1. Introduction Transcripts of debates held in the United Kingdom (UK) Parliament are publicly and freely available and provide access to the opinions and attitudes of Members of Parlia- ment (MPs) and the political parties they represent towards the most important topics facing society and its citizens, as well as potential insights into the parliamentary demo- cratic process. As a result, they are of interest to the politi- cians themselves, the media, social scientists and histori- ans, and any members of the public who wish to scrutinise the activities of their elected representatives. However, due to the large quantity and complexity of the material, pro- cessing the transcripts and analysing the positions taken by the speakers can be a difficult and overwhelming task for humans (Salah, 2014; Thomas et al., 2006). There has therefore been considerable interest in applying automatic sentiment analysis methods to political debates from legislatures such as the US Congress (Bhatia and P, 2018; Ji and Smith, 2017) and the UK Parliament (Bha- van et al., 2019; Salah et al., 2013). However, in com- parison with those currently used in other domains, such as product reviews or social media, which can run into the hundreds of thousands of labelled examples, the datasets currently available for this task are relatively small (see Table 1). As current neural network and embedding-based state-of-the-art sentiment analysis methods tend to benefit from significantly larger datasets, there is a need to develop more extensive …

Content truncated to 500 words