Policy Preference Detection in Parliamentary Debate Motions

Proceedings of the 23rd Conference on Computational Natural Language Learning, pages 249–259 Hong Kong, China, November 3-4, 2019. c⃝2019 Association for Computational Linguistics 249 Policy Preference Detection in Parliamentary Debate Motions Gavin Abercrombie Department of Computer Science University of Manchester gavin.abercrombie@ manchester.ac.uk Riza Batista-Navarro Department of Computer Science University of Manchester riza.batista@ manchester.ac.uk Federico Nanni Data and Web Science Group University of Mannheim federico@ informatik.uni-mannheim.de Simone Paolo Ponzetto Data and Web Science Group University of Mannheim simone@ informatik.uni-mannheim.de Abstract Debate motions (proposals) tabled in the UK Parliament contain information about the stated policy preferences of the Members of Parliament who propose them, and are key to the analysis of all subsequent speeches given in response to them. We attempt to automat- ically label debate motions with codes from a pre-existing coding scheme developed by po- litical scientists for the annotation and analy- sis of political parties’ manifestos. We develop annotation guidelines for the task of applying these codes to debate motions at two levels of granularity and produce a dataset of manually labelled examples. We evaluate the annotation process and the reliability and utility of the labelling scheme, finding that inter-annotator agreement is comparable with that of other studies conducted on manifesto data. More- over, we test a variety of ways of automati- cally labelling motions with the codes, rang- ing from similarity matching to neural classifi- cation methods, and evaluate them against the gold standard labels. From these experiments, we note that established supervised baselines are not always able to improve over simple lex- ical heuristics. At the same time, we detect a clear and evident benefit when employing BERT, a state-of-the-art deep language repre- sentation model, even in classification scenar- ios with over 30 different labels and limited amounts of training data. 1 Introduction Commonly known as the Hansard record, tran- scripts of debates that take place in the House of Commons of the United Kingdom (UK) Parlia- ment are of interest to scholars of political sci- ence as well as the media and members of the public who wish to monitor the actions of their elected representatives. Debate motions (the pro- posals tabled for debate) are expressions of the policy positions taken by the governments, politi- cal parties, and individual Members of Parliament (MPs) who propose them. As all speeches given and all votes cast in the House are responses to one of these proposals, the motions are key to any understanding and analysis of the opinions and po- sitions expressed in the subsequent speeches given in parliamentary debates. By definition, debate motions convey the stated policy preferences of the MPs or parties who pro- pose them. They therefore express polarity— positive or negative—towards some target, such as a piece of legislation, policy, or state of affairs. As noted by Thomas et al. (2006), the polarity of a debate proposal can strongly affect the language used by debate participants to either support or oppose it, effectively acting as a polarity shifter on the ensuing speeches. Analysis of …

Content truncated to 500 words