Columbia at SemEval-2019 Task 7: Multi-task Learning for Stance Classification and Rumour Verification

Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval-2019), pages 1110–1114 Minneapolis, Minnesota, USA, June 6–7, 2019. ©2019 Association for Computational Linguistics 1110 Columbia at SemEval-2019 Task 7: Multi-task Learning for Stance Classification and Rumour Verification Zhuoran Liu* Shivali Goel* Department of Computer Science, Columbia University {zhuoran.liu, shivali.goel, my2541}@columbia.edu Mukund Yelahanka Raghuprasad Smaranda Muresan Data Science Institute, Columbia University sm761@columbia.edu Abstract The paper presents Columbia team’s participa- tion in the SemEval 2019 Shared Task 7: Ru- mourEval 2019. Detecting rumour on social networks has been a focus of research in re- cent years. Previous work suffered from data sparsity, which potentially limited the appli- cation of more sophisticated neural architec- ture to this task. We mitigate this problem by proposing a multi-task learning approach to- gether with language model fine-tuning. Our attention-based model allows different tasks to leverage different level of information. Our system ranked 6th overall with an F1-score of 36.25 on stance classification and F1 of 22.44 on rumour verification. 1 Introduction The ubiquity of social media is allowing unverified news and rumours to spread easily. Efforts have been made to explore automated methods for ru- mour detection and verification (Derczynski et al., 2017; Zubiaga et al., 2016, 2018), and has shown promising potential to tackle this issue at scale. RumourEval 2019 Shared Task 7 tackles the problem of predicting the veracity of rumours and stance of replies. It consists of two subtasks: task A (SDQC), in which stance (support, deny, query- ing, comment) of responses to a rumourous state- ment are predicted, and task B (verification), in which the statement’s veracity is to be predicted. Size of training data provided for Task A is 5,217 and for Task B is 327. In this paper, we proposed several methods to alleviate data sparsity and unleash the power of so- phisticated neural models: 1. Jointly learning to perform rumour verifica- tion and stance detection. Training a neural * Equal contribution. network on limited amount of data for a sin- gle task is hard, especially in a sentence clas- sification task. This is because of the weak supervision signal caused by the information asymmetry between the source text and the target labels. With supervision signal from multiple tasks, a neural network can exploit information in the training data more thor- oughly. 2. Using self-attention. To predict the stance of a post, we want to selectively pay attention to some other posts that are relevant to this post. We use a QKV-style attention (Query, Key, Value) (Vaswani et al., 2017) to sum- marize the post context into a single vector (where in practice one attention head is usu- ally enough). In addition, we use representa- tions at different levels for different tasks. 3. Using language model fine-tuning for stance classification. We use the Universal Lan- guage Model Fine-tuning (ULMFiT) (Ruder and Howard, 2018) to improve our stance classifier. We begin with a generic language model trained on the Wikitext 103 dataset (Merity et al., 2016). This dataset consists of a …

Content truncated to 500 words