Optimising Twitter-based Political Election Prediction with Relevance andSentiment Filters

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 6158–6165 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 6158 Optimising Twitter-based Political Election Prediction with Relevance and Sentiment Filters Eric Sanders, Antal van den Bosch CLS/CLST, Radboud University, Nijmegen KNAW Meertens Institute, Amsterdam e.sanders@let.ru.nl, antal.van.den.bosch@meertens.knaw.nl Abstract We study the relation between the number of mentions of political parties in the last weeks before the elections and the election results. In this paper we focus on the Dutch elections of the parliament in 2012 and for the provinces (and the senate) in 2011 and 2015. With raw counts, without adaptations, we achieve a mean absolute error (MAE) of 2.71% for 2011, 2.02% for 2012 and 2.89% for 2015. A set of over 17,000 tweets containing political party names were annotated by at least three annotators per tweet on ten features denoting communicative intent (including the presence of sarcasm, the message’s polarity, the presence of an explicit voting endorsement or explicit voting advice, etc.). The annotations were used to create oracle (gold-standard) filters. Tweets with or without a certain majority annotation are held out from the tweet counts, with the goal of attaining lower MAEs. With a grid search we tested all combinations of filters and their responding MAE to find the best filter ensemble. It appeared that the filters show markedly different behaviour for the three elections and only a small MAE improvement is possible when optimizing on all three elections. Larger improvements for one election are possible, but result in deterioration of the MAE for the other elections. Keywords: Large Scale Annotation, Election Prediction, Relevance Filters, Sentiment Analysis 1. Introduction Several papers have reported on the relation of mentions of political parties in tweets and the outcome of the elections in several countries (Jungherr, 2015). There is broadly re- ported empirical evidence for a strong correlation between the two. This indicates that Twitter might be a good pre- dictor of election results. However, some papers offer criti- cal counterpoints to this claim (Gayo-Avello, 2013). Major points of critique involve the mismatch of demographics in the Twitter and election population, no or erroneous use of sentiment analysis, and ignoring the contents of the tweets. In a previous study we attempted to tackle the demographic mismatch (Sanders et al., 2016). In the present study we at- tempt to take into account the sentiment and contents of the tweets. To this purpose we set up a webtool for annotation of tweets in which political parties are mentioned. This resulted in a set of over 17,000 tweets classified by their communicative intent and sentiment: we asked annotators to identify sarcasm, explicit endorsements, etc. These an- notations were used to create filters for exclusion of tweets with certain (combinations of) features, e.g. for removing all sarcastic tweets, or for removing all tweets in which the person posting the tweet explicitly states that he or she will not vote for a particular party. By applying filters to …

Content truncated to 500 words