Multimodal Semi-supervised Learning for Disaster Tweet Classification

Proceedings of the 29th International Conference on Computational Linguistics, pages 2711–2723 October 12–17, 2022. 2711 Multimodal Semi-supervised Learning for Disaster Tweet Classification Iustin Sirbu University Politehnica of Bucharest iustin.sirbu@upb.ro Tiberiu Sosea University of Illinois Chicago tsosea2@uic.edu Cornelia Caragea University of Illinois Chicago cornelia@uic.edu Doina Caragea Kansas State University dcaragea@ksu.edu Traian Rebedea University Politehnica of Bucharest traian.rebedea@upb.ro Abstract During natural disasters, people often use so- cial media platforms, such as Twitter, to post information about casualties and damage pro- duced by disasters. This information can help relief authorities gain situational awareness in nearly real time, and enable them to quickly distribute resources where most needed. How- ever, annotating data for this purpose can be burdensome, subjective and expensive. In this paper, we investigate how to leverage the co- pious amounts of unlabeled data generated on social media by disaster eyewitnesses and af- fected individuals during disaster events. To this end, we propose a semi-supervised learn- ing approach to improve the performance of neural models on several multimodal disaster tweet classification tasks. Our approach shows significant improvements, obtaining up to 7.7% improvements in F-1 in low-data regimes and 1.9% when using the entire training data. We make our code and data publicly available.1 1 Introduction The upswing of text and image sharing on social media platforms, such as Twitter, during mass emer- gency situations has led to numerous opportunities to gain timely access to valuable information that can help disaster relief authorities act quicker and more efficiently. Specifically, as a disaster unfolds, information shared on social media can provide insights into the infrastructure and utility damage, casualties, and missing people. Recent studies have focused on collecting and manually annotating dis- aster data with respect to such situational aware- ness categories, followed by training machine learn- ing classifiers to automatically identify situational awareness information, useful for relief operations (Alam et al., 2018b; Ashktorab et al., 2014). However, disaster events produce large amounts of user-generated data, of which only a small frac- 1https://github.com/iustinsirbu13/multimodal-ssl-for- disaster-tweet-classification tion can be annotated, due to the time-sensitive nature of the problem, together with high annota- tion costs, and also inherent subjectivity associated with annotating data (e.g., tweets). To address this limitation, we propose a semi- supervised multimodal approach that can lever- age the copious amounts of unlabeled data to im- prove the performance on various multimodal tasks. Specifically, we extend the FixMatch (Sohn et al., 2020) algorithm proposed for semi-supervised im- age classification to a multimodal setting. To ac- count for subjective annotations and potentially overlapping labels, we use soft pseudo-labels in- stead of the original hard pseudo-labels. We apply the adapted FixMatch to the CrisisMMD labeled dataset and tasks (Alam et al., 2018b), to improve the performance of supervised baselines through the use of unlabeled data. We use 122K unlabeled tweets, containing both text and images, collected automatically using text queries about disasters that occurred during the year of 2017. Experimental results show that our proposed ap- proach produces performance improvements on all three CrisisMMD tasks in various …

Content truncated to 500 words