DFKI-DKT at SemEval-2017 Task 8: Rumour Detection and Classification using Cascading Heuristics

Proceedings of the 11th International Workshop on Semantic Evaluations (SemEval-2017), pages 486–490, Vancouver, Canada, August 3 - 4, 2017. c⃝2017 Association for Computational Linguistics DFKI-DKT at SemEval-2017 Task 8: Rumour Detection and Classification using Cascading Heuristics Ankit Kumar Srivastava DFKI GmbH Alt-Moabit 91c, 10559 Berlin, DE ankit.srivastava@dfki.de Georg Rehm DFKI GmbH Alt-Moabit 91c, 10559 Berlin, DE georg.rehm@dfki.de Julian Moreno Schneider DFKI GmbH Alt-Moabit 91c, 10559 Berlin, DE julian.moreno_schneider@dfki.de Abstract We describe our submissions for SemEval-2017 Task 8, Determining Rumour Veracity and Support for Ru- mours. The Digital Curation Technologies (DKT) (Rehm and Sasaki, 2016, 2015) team at the German Research Center for Artificial Intelligence (DFKI) participated in two subtasks: Subtask A (determining the stance of a message) and Subtask B (determining veracity of a message, closed variant). In both cases, our implementa- tion consisted of a Multivariate Logistic Regression (Maximum Entropy) classifier coupled with hand-written patterns and rules (heuristics) applied in a post-process cascading fashion. We provide a detailed analysis of the system performance and report on variants of our systems that were not part of the official submission. 1 Introduction In today’s digital age, the social, political and eco- nomic relevance of online media and online con- tent is becoming more and more relevant. Accord- ingly, the task of analysing and determining the veracity of online content is receiving a growing amount of attention by the NLP community. The ability to detect whether a piece of news is fake or not, and to do so automatically, is a very timely language technology application (Zubiaga and Ji, 2014). Through these shared tasks, we intend to address which linguistic and contextual features characterise a rumour. SemEval2017 Task 8 (Derczynski et al., 2017) provided all participants with a dataset consisting of tweets in response to breaking news stories. It contains conversations responding to rumourous tweets. These tweets have been annotated for sup- port, deny, query or comment (SDQC). The com- petition consisted of two subtasks: • Subtask A: Determining whether response tweets support, deny, query or comment (SDQC) on rumours (source tweet) • Subtask B: Given a tweet, determine whether the statement is true or false (i. e., a rumour). This subtask featured two vari- ants: closed (determining veracity from the tweet alone) and open (determining veracity from additional context). We participated in the closed task. Our approach to both subtasks involved extract- ing relevant features from the provided data and training a classifier followed by a set of heuris- tics implemented in a cascading decision tree style (Minguillon, 2002). These rules, applied as a post- process, help induce a better mapping from classi- fication results to rumour categorisation and verac- ity detection because they take into account spe- cific features characterising a particular class. In this paper we seek to answer two questions using Rumour Detection and Classification as a case-study: • Which features comprise the set of post- process rules? • What is the optimal technique to implement these heuristics (cascading order)? This paper is structured as follows. Section 2 …

Content truncated to 500 words