P-Stance: A Large Dataset for Stance Detection in Political Domain

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 2355–2365 August 1–6, 2021. ©2021 Association for Computational Linguistics 2355 P-Stance: A Large Dataset for Stance Detection in Political Domain Yingjie Li♣ Tiberiu Sosea♣ Aditya Sawant♣ Ajith Jayaraman Nair♣ Diana Inkpen♦ Cornelia Caragea♣ ♣Computer Science, University of Illinois at Chicago ♦Electrical Engineering and Computer Science, University of Ottawa ♣{yli300,tsosea2,asawan8,anair34,cornelia}@uic.edu ♦{diana.inkpen}@uottawa.ca Abstract Stance detection determines whether the au- thor of a text is in favor of, against or neu- tral to a specific target and provides valuable insights into important events such as presi- dential election. However, progress on stance detection has been hampered by the absence of large annotated datasets. In this paper, we present P-STANCE, a large stance detection dataset in the political domain, which contains 21,574 labeled tweets. We provide a detailed description of the newly created dataset and develop deep learning models on it. Our best model achieves a macro-average F1-score of 80.53%, which we improve further by using semi-supervised learning. Moreover, our P- STANCE dataset can facilitate research in the fields of cross-domain stance detection such as cross-target stance detection where a classifier is adapted from a different but related target. We publicly release our dataset and code.1 1 Introduction Nowadays, people often express their stances to- ward specific targets (e.g., political events or fig- ures, religion, or abortion) on social media. These opinions can provide valuable insights into impor- tant events, e.g., presidential election. The goal of the stance detection task is to determine whether the author of a piece of text is in favor of, against, or neutral toward a specific target (Mohammad et al., 2016b; Küçük and Can, 2020; ALDayel and Magdy, 2021). Twitter as a social platform has pro- duced a large quantity of user-generated content, which has become a rich source for mining useful information about various topics such as presiden- tial election. Political figures, who usually receive considerable attention and involve themselves in a large number of political events, are great targets to study stance detection. Therefore, detecting the 1https://github.com/chuchun8/PStance stance expressed toward political figures on Twitter has drawn a lot of attention in the NLP community (Mohammad et al., 2016a; Sobhani et al., 2017; Darwish et al., 2017). Even though stance detection has received a lot of attention, the annotated data are usually limited, which poses strong challenges to supervised mod- els. Moreover, a limitation of existing datasets is that explicit mentions of targets and surface-level lexical cues that may expose the stance can be widely observed in the data (Mohammad et al., 2016a; Sobhani et al., 2017; Swami et al., 2018; Darwish et al., 2018; Conforti et al., 2020b; Lai et al., 2020), which means a model can detect the stance without extracting effective representations for the meanings of sentences (i.e., their lexical and compositional semantics). Another limitation of existing datasets, especially the datasets built on social media, is that the average length of tweets is short, which indicates that the data in these previ- ous datasets are …

Content truncated to 500 words