The Role of Context in Vaccine Stance Prediction for Twitter Users

The Role of Context in Vaccine Stance Prediction for Twitter Users Aleney Khoo1,2 Maciej Rybinski1 Sarvnaz Karimi1 Adam Dunn2 1CSIRO Data61, Sydney, Australia 2The University of Sydney, Sydney Australia {firstname.lastname}@csiro.au adam.dunn@sydney.edu.au Abstract Public expression of vaccine related sentiment on social media platforms can be used in infor- mation surveillance applications to gain insight into vaccine hesitancy and its spread. Effective identification of vaccine-negative content con- stitutes one of the most fundamental building blocks in such applications. Here, we investigate the role of users’ previous vaccine-related posts, in the capacity of stance text classifiers to detect vaccine-negative con- tent. We conduct experiments on a dataset of over 7K tweets manually labeled for vaccina- tion stance captured between 2017 and 2019, with unlabeled historical data. Our results indicate that incorporating user- generated-context improves stance detection. It also bridges the effectiveness gap between simple linear models and state-of-the art text classifiers, highlighting the importance of data capture strategy to the downstream task. 1 Introduction Vaccine hesitancy is defined as delayed acceptance or refusal of vaccines despite their availability. It is believed that vaccine hesitancy and refusal may be amplified by the spread of vaccine-negative content on social media (Raballo et al., 2022). Stance detection is a process of identification of speaker’s judgment or standpoint towards a given proposition (Biber and Finegan, 1988; Mohammad et al., 2017). It is often modelled as a classifica- tion task of assigning an against, for or neural label to a text for a given target. In the context of social media, detecting anti-vaccine content can be seen as a building block fundamental to imple- mentation of mitigation and monitoring strategies, understanding fears and concerns expressed in the public discourse (Mitra et al., 2021). In this prob- lem, vaccine hesitancy is considered the target for stance detection. In this paper we simplify the problem of stance detection by modelling it as a binary text classifi- cation task. In this set-up we investigate the im- pact of including users’ historic tweets in detecting vaccine-negative utterances. We conduct experi- ments on a dataset of over 7K tweets manually labelled for vaccine stance captured between 2017 and 2019, which contains unlabelled historical data from the authors of the labelled tweets. Our work focuses on the impact of the availability of his- toric user-generated content on effectiveness on the downstream text classification task. We com- pare Transformer-based models, which allow for modelling the historical context in a cross-encoder, with traditional linear models incorporating these historical tweets in a Bag-of-Words (BoW) repre- sentation encompassing the labelled utterance. 2 Related Work Our study relates to the literature on using social media for public health and, in particular, for de- tection of vaccination hesitancy. It also relates to Twitter, and other social media, text classification, sentiment and stance detection. Social Media for Public Health Natural Lan- guage Processing (NLP) techniques for social me- dia have been leveraged in different public health applications (Paul and Dredze, 2017; Conway et al., 2019), including for mental health (Calvo et …

Content truncated to 500 words