Hierarchical Structured Model for Fine-to-Coarse Manifesto Text Analysis

Proceedings of NAACL-HLT 2018, pages 1964–1974 New Orleans, Louisiana, June 1 - 6, 2018. c⃝2018 Association for Computational Linguistics Hierarchical Structured Model for Fine-to-coarse Manifesto Text Analysis Shivashankar Subramanian Trevor Cohn Timothy Baldwin School of Computing and Information Systems The University of Melbourne shivashankar@student.unimelb.edu.au {t.cohn,tbaldwin}@unimelb.edu.au Abstract Election manifestos document the intentions, motives, and views of political parties. They are often used for analysing a party’s fine- grained position on a particular issue, as well as for coarse-grained positioning of a party on the left–right spectrum. In this paper we propose a two-stage model for automatically performing both levels of analysis over man- ifestos. In the first step we employ a hierar- chical multi-task structured deep model to pre- dict fine- and coarse-grained positions, and in the second step we perform post-hoc calibra- tion of coarse-grained positions using proba- bilistic soft logic. We empirically show that the proposed model outperforms state-of-art approaches at both granularities using mani- festos from twelve countries, written in ten dif- ferent languages. 1 Introduction The adoption of NLP methods has led to signif- icant advances in the field of computational so- cial science (Lazer et al., 2009), including politi- cal science (Grimmer and Stewart, 2013). Among a myriad of data sources, election manifestos are a core artifact in political analysis. One of the most widely used datasets by political scientists is the Comparative Manifesto Project (CMP) dataset (Volkens et al., 2017), which contains manifestos in various languages, covering over 1000 parties across 50 countries, from elections dating back to 1945. In CMP, a subset of the manifestos has been manually annotated at the sentence-level with one of 57 political themes, divided into 7 major cat- egories.1 Such categories capture party posi- tions (FAVORABLE, UNFAVORABLE or NEITHER) 1https://manifesto-project.wzb.eu/ coding_schemes/mp_v5 on fine-grained policy themes, and are also use- ful for downstream tasks including calculating manifesto-level (policy-based) left–right position scores (Budge et al., 2001; Lowe et al., 2011; D¨aubler and Benoit, 2017). An example sentence from the Green Party of England and Wales 2015 election manifesto where they take an UNFAVOR- ABLE position on MILITARY is: We would: Ensure that ... less is spent on military research. Elsewhere, they take a FAVORABLE position on WELFARE STATE: Double Child Benefit. Such manual annotations are labor-intensive and prone to annotation inconsistencies (Mikhaylov et al., 2012). In order to overcome these challenges, supervised sentence classification ap- proaches have been proposed (Verberne et al., 2014; Subramanian et al., 2017). Other than the sentence-level labels, the man- ifesto text also has a document-level score that quantifies its position on the left–right spectrum. Different approaches have been proposed to de- rive this score, based on alternate definitions of “left–right” (Slapin and Proksch, 2008; Benoit and Laver, 2007; Lo et al., 2013; D¨aubler and Benoit, 2017). Among these, the RILE index is the most widely adopted (Merz et al., 2016; Jou and Dalton, 2017), and has been shown to correlate highly with other popular scores (Lowe et al., 2011). RILE is defined as the difference between …

Content truncated to 500 words