SemEval-2019 Task 7: RumourEval, Determining Rumour Veracity and Support for Rumours

Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval-2019), pages 845–854 Minneapolis, Minnesota, USA, June 6–7, 2019. ©2019 Association for Computational Linguistics 845 RumourEval 2019: Determining Rumour Veracity and Support for Rumours Genevieve Gorrell1 and Elena Kochkina3,4 and Maria Liakata3,4 and Ahmet Aker1 and Arkaitz Zubiaga5 and Kalina Bontcheva1 and Leon Derczynski2 1University of Sheffield, UK (g.gorrell,a.aker,k.bontcheva)@sheffield.ac.uk 2IT University of Copenhagen, Denmark ld@itu.dk 3University of Warwick, UK (e.kochkina,m.liakata)@warwick.ac.uk 4Alan Turing Institute, UK 5Queen Mary University of London, UK a.zubiaga@qmul.ac.uk Abstract Since the first RumourEval shared task in 2017, interest in automated claim validation has greatly increased, as the danger of “fake news” has become a mainstream concern. However automated support for rumour verifi- cation remains in its infancy. It is therefore im- portant that a shared task in this area continues to provide a focus for effort, which is likely to increase. Rumour verification is characterised by the need to consider evolving conversations and news updates to reach a verdict on a ru- mour’s veracity. As in RumourEval 2017 we provided a dataset of dubious posts and ensu- ing conversations in social media, annotated both for stance and veracity. The social me- dia rumours stem from a variety of breaking news stories and the dataset is expanded to include Reddit as well as new Twitter posts. There were two concrete tasks; rumour stance prediction and rumour verification, which we present in detail along with results achieved by participants. We received 22 system submis- sions (a 70% increase from RumourEval 2017) many of which used state-of-the-art methodol- ogy to tackle the challenges involved. 1 Introduction 1.1 Background Since the first RumourEval shared task in 2017 (Derczynski et al., 2017), interest in automated verification of rumours has deepened, as research has demonstrated the potential impact of false claims on important political outcomes (Allcott and Gentzkow, 2017). Living in a “post-truth world”, in which perceived truth can matter more than actual truth (Dale, 2017), the dangers posed by unchecked market forces and cheap platforms, as well as poor ability by many readers to discern credible information, are evident. As a result the importance of educating young people about crit- ical thinking is increasingly emphasised.1. More- over the European Commission’s High Level Ex- pert Group on Fake News provides tools to em- power users and journalists to tackle disinforma- tion as one of the five pillars of their recommended approach.2 Platforms are increasingly motivated to engage with the problem of damaging con- tent that appears on them, as society moves to- ward a consensus regarding their level of respon- sibility. Independent fact checking efforts, such as Snopes3, Full Fact4, Chequeado5, are also be- coming valued resources (Konstantinovskiy et al., 2018). Zubiaga et al. (2018) present an exten- sive list of projects. Effort so far is often manual, and struggles to keep up with the large volume of online material. Within NLP research the tasks of stance clas- sification of news articles and social media posts and the creation of systems to automatically iden- …

Content truncated to 500 words