CLEARumor at SemEval-2019 Task 7: ConvoLving ELMo Against Rumors

Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval-2019), pages 1105–1109 Minneapolis, Minnesota, USA, June 6–7, 2019. ©2019 Association for Computational Linguistics 1105 CLEARumor at SemEval-2019 Task 7: ConvoLving ELMo Against Rumors Ipek Baris1,∗and Lukas Schmelzeisen1,∗and Steffen Staab1, 2 1Institute for Web Science and Technologies (WeST), University of Koblenz-Landau, Germany 2Web and Internet Science Group (WAIS), University of Southampton, United Kingdom ibaris@uni-koblenz.de, lukas@uni-koblenz.de, staab@uni-koblenz.de Abstract This paper describes our submission SemEval-2019 Task 7: RumourEval: Deter-to mining Rumor Veracity and Support for Ru- mors. We participated in both subtasks. The goal of subtask A is to classify the type of interaction between a rumorous social media post and a reply post as support, query, deny, or comment. The goal of subtask B is to pre- dict the veracity of a given rumor. For subtask A, we implement a CNN-based neural archi- tecture using ELMo embeddings of post text combined with auxiliary features and achieve a F1-score of 44.6%. For subtask B, we em- ploy a MLP neural network leveraging our es- timates for subtask A and achieve a F1-score of 30.1% (second place in the competition). We provide results and analysis of our system per- formance and present ablation experiments. 1 Introduction Online social media has changed the way of com- municating and disseminating media content and opinions, but also paved the way for spreading false or unverified rumors. RumourEval 2019 (Gorrell et al., 2019) pro- vides a dataset of labelled threads from Twitter and Reddit where each source post mentions a rumor. Subtask A (SDQC) consists of deciding for each post in a thread whether it is in a support, deny, query, or comment relation to the rumor. The goal of subtask B (Verification) it to classify the verac- ity of the rumor as true, false, or unverified. Fig- ure 1 clarifies our terminology and the tasks. Automated rumor classification is a challenging task as there is no definite evidence (e.g., autho- rized confirmation). In its absence, stance analysis is a useful approach. Systems that employ neural network architectures showed promising results in RumourEval 2017 (Derczynski et al., 2017), with ∗The first two authors contributed equally. Figure 1: An example Twitter thread from the training dataset with SDQC labels for each post and a veracity label for the thread’s source post. Any post that does not reply to another is a source post. Reply posts can be direct replies (replies to a source post) or nested replies (replies that reply to another reply post). A thread is the set containing a source post and all its reply posts. the LSTM-based sequential model of Kochkina et al. (2017) performing best. In this paper, we describe our approach CLEARumor (ConvoLving ELMo Against Ru- mors) for solving both subtasks and provide em- pirical results and ablation experiments of our ar- chitecture. We make our PyTorch-based imple- mentation and trained models publicly available1. 2 System Description After preprocessing the post text (Section 2.1) and embedding it with ELMo (Section 2.2), our ar- chitecture …

Content truncated to 500 words