Joint Sentence-Document Model for Manifesto Text Analysis

Joint Sentence–Document Model for Manifesto Text Analysis Shivashankar Subramanian Trevor Cohn Timothy Baldwin Julian Brooke School of Computing and Information Systems University of Melbourne shivashankar@student.unimelb.edu.au, {t.cohn, tbaldwin, julian.brooke}@unimelb.edu.au Abstract Election manifestos document the inten- tions, motives, and views of political par- ties. They are often used for analysing party policies and positions on various is- sues, as well as for quantifying a party’s position on the left–right spectrum. In this paper we propose a model for auto- matically predicting both types of anal- ysis from manifestos, based on a joint sentence–document approach which per- forms both sentence-level thematic classi- fication and document-level position quan- tification. Our method handles text in multiple languages, via the use of mul- tilingual vector-space embeddings. We empirically show that the proposed joint model performs better than state-of-art ap- proaches for the document-level task and provides comparable performance for the sentence level task, using manifestos from thirteen countries, written in six different languages. 1 Introduction Election manifestos are a core artifact in political text analysis. One of the widely used datasets by political scientists is the Comparative Manifesto Project (CMP) dataset, initiated by Volkens et al. (2011), that collects party manifestos from elec- tions in many countries around the world. The goal of the project is to provide a large data collec- tion to support political studies on electoral pro- cesses. A sub-part of the manifestos has been manually annotated at the sentence-level with one of over fifty fine-grained political themes, divided into 7 coarse-grained topics (see Table 5). These are important because it can be seen as party posi- tions on fine-grained policy themes and also the coded text can be used for various downstream tasks (Lowe et al., 2011). While manual annota- tions are very useful for political analyses, they come with two major drawbacks. First, it is very time-consuming and labor-intensive to manually annotate each sentence with the correct category from a complex annotation scheme. Secondly, coder preferences towards particular categories might lead to annotation inconsistencies and af- fect comparability between manifestos annotated by different coders (Mikhaylov et al., 2012). In or- der to overcome these challenges, fine and coarse- level manifesto sentence classification was ad- dressed using supervised machine learning tech- niques (Verberne et al., 2014; Zirn et al., 2016). Nonetheless, manually-coded manifestos remain the crucial data source for studies in computational political science (Lowe et al., 2011; Nanni et al., 2016). Other than the sentence-level labels, the mani- festo text also has document-level signals, which quantify its position on the left–right spectrum (Slapin and Proksch, 2008). Though sentence- level classification and document-level quantifica- tion tasks are inter-dependent, existing work han- dles them separately. We instead propose a joint approach to model the two tasks together. Overall, the contributions of this work are as follows: • we empirically study the utility of multi- lingual embeddings for cross-lingual mani- festo text analysis — at the sentence (for 57- class classification) and document-levels (for RILE score regression) • we evaluate the effectiveness of modelling …

Content truncated to 500 words