Tribrid: Stance Classification with Neural Inconsistency Detection

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6831–6843 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 6831 Tribrid: Stance Classification with Neural Inconsistency Detection Song Yang Department of Computer Science Vrije Universiteit Amsterdam The Netherlands s.yang@student.vu.nl Jacopo Urbani Department of Computer Science Vrije Universiteit Amsterdam The Netherlands jacopo@cs.vu.nl Abstract We study the problem of performing automatic stance classification on social media with neu- ral architectures such as BERT. Although these architectures deliver impressive results, their level is not yet comparable to the one of hu- mans and they might produce errors that have a significant impact on the downstream task (e.g., fact-checking). To improve the perfor- mance, we present a new neural architecture where the input also includes automatically gen- erated negated perspectives over a given claim. The model is jointly learned to make simul- taneously multiple predictions, which can be used either to improve the classification of the original perspective or to filter out doubtful pre- dictions. In the first case, we propose a weakly supervised method for combining the predic- tions into a final one. In the second case, we show that using the confidence scores to re- move doubtful predictions allows our method to achieve human-like performance over the re- tained information, which is still a sizable part of the original input. 1 Introduction The spreading of unverified claims on social me- dia is an important problem that affects our soci- ety at multiple levels (Vlachos and Riedel, 2014; Ciampaglia et al., 2015; Hassan et al., 2017). A valuable asset that we can exploit to fight this prob- lem is the set of perspectives that people publish about such claims. These perspectives reveal the users’ stance and this information can be used to help an automated framework to determine more accurately the veracity of rumors (Castillo et al., 2011; Bourgonje et al., 2017). The stance can be generally categorized either as supportive or opposing. For instance, consider the claim “The elections workers in Wisconsin illegally altered absentee ballot envelopes”. A tweet with a supportive stance is “The number of people who took part in the election in Wisconsin exceeded the total number of registered voters” while one with an opposed stance is “An extra zero was added as votes accidentally but it was quickly fixed after state officials noticed it”. Being able to automatically classify the stance is necessary for dealing with the large volume of data that flows through social networks. To this end, ear- lier solutions relied on linguistic features, such as n- grams, opinion lexicons, and sentiment (Somasun- daran and Wiebe, 2009; Anand et al., 2011; Hasan and Ng, 2013; Sridhar et al., 2015) while more recent methods additionally include features that we can extract from networks like Twitter (Chen and Ku, 2016; Lukasik et al., 2016; Sobhani et al., 2017; Kochkina et al., 2017). The current state-of- the-art relies on advanced neural architectures such as BERT (Devlin et al., 2019) and returns remark- able performance. For instance, …

Content truncated to 500 words