“We will Reduce Taxes” - Identifying Election Pledges with Language Models

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 3406–3419 August 1–6, 2021. ©2021 Association for Computational Linguistics 3406 “We will Reduce Taxes” Identifying Election Pledges with Language Models Tommaso Fornaciari Universit`a Bocconi fornaciari@unibocconi.it Julia Runeson University of Gothenburg julia.runeson@gu.se Dirk Hovy Universit`a Bocconi dirk.hovy@unibocconi.it Robert Thomson Monash University robert.thomson@monash.edu Elin Naurin University of Gothenburg elin.naurin@pol.gu.se Pankaj Adhikari Monash University pankaj.adhikari@monash.edu Abstract In an election campaign, political parties pledge to implement various projects–should they be elected. But do they follow through? To track election pledges from parties’ election manifestos, we need to distinguish between pledges and general statements. In this paper, we use election manifestos of Swedish and In- dian political parties to learn neural models that distinguish actual pledges from generic political positions. Since pledges might vary by election year and party, we implement a Multi-Task Learning (MTL) setup, predicting election year and manifesto’s party as auxil- iary tasks. Pledges can also span several sen- tences, so we use hierarchical models that in- corporate contextual information. Lastly, we evaluate the models in a Zero-Shot Learning (ZSL) framework across countries and lan- guages. Our results indicate that year and party have predictive power even in ZSL, while con- text introduces some noise. We finally discuss the linguistic features of pledges. 1 Introduction Before any election, political parties publish man- ifestos that summarize their pledges to the voters. The exact nature of those pledges varies. A single- issue party might campaign on the same promise year after year, but most parties will adapt to the shifting trends and needs of the electorate. How- ever, there is a difference between pledging and fulfilling. Political scientists are highly interested in whether pledges were fulfilled, a question that is gaining a growing interest in the broader sci- entific community (Naurin et al., 2019). Several approaches exist, but they are primarily confined to manual analysis of individual countries or elec- tions. They indicate that governmental parties mostly fulfill their election pledges (Naurin et al., 2019; Thomson et al., 2017). However, there are too many elections worldwide to analyze all cam- paign pledges manually. We need automated ways to identify pledges and hold governments account- able systematically. Checking whether a pledge was fulfilled still re- quires manual work by trained political scientists, but the first step–identifying pledges–is a problem very much made for NLP, for at least two reasons. First, NLP can automate pledge identification to distinguish pledges from irrelevant content. This allows the study of pledge fulfillment at scale. An average election manifesto in our corpus has 418 sentences, but only 118 of them (27.5%) will con- tain a pledge. The rest is filler material. It takes several days to train an annotator, who then spends around 6-8 hours on a single manifesto, to iden- tify those 27.5% of pledges. Cutting down on this laborious first part frees up time to focus on the more complex issue of determining whether those pledges were fulfilled. Second, NLP methods can help us understand the linguistic style …

Content truncated to 500 words