Twitter-COMMs: Detecting Climate, COVID, and Military Multimodal Misinformation

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1530 - 1549 July 10-15, 2022 ©2022 Association for Computational Linguistics Twitter-COMMs: Detecting Climate, COVID, and Military Multimodal Misinformation Giscard Biamby ∗ Grace Luo ∗ Trevor Darrell Anna Rohrbach University of California, Berkeley {gbiamby, graceluo, trevordarrell, anna.rohrbach}@berkeley.edu Abstract Detecting out-of-context media, such as “mis- captioned” images on Twitter, is a relevant problem, especially in domains of high pub- lic significance. In this work we aim to develop defenses against such misinformation for the topics of Climate Change, COVID-19, and Mil- itary Vehicles. We first present a large-scale multimodal dataset with over 884k tweets rel- evant to these topics. Next, we propose a de- tection method, based on the state-of-the-art CLIP model, that leverages automatically gen- erated hard image-text mismatches. While this approach works well on our automatically con- structed out-of-context tweets, we aim to vali- date its usefulness on data representative of the real world. Thus, we test it on a set of human- generated fakes created by mimicking in-the- wild misinformation. We achieve an 11% de- tection improvement in a high precision regime over a strong baseline. Finally, we share in- sights about our best model design and analyze the challenges of this emerging threat. 1 Introduction Out-of-context images are a popular form of mis- information where an image is miscaptioned to support a false claim (Fazio, 2020). Such image re- purposing is extremely cheap yet can be as damag- ing as more sophisticated fake media. In this work we focus on domains important for society and na- tional security, where implications of inexpensive yet effective misinformation can be immense. Specifically, we analyze multimodal Twitter posts that are of significant public interest, re- lated to topics of COVID-19, Climate Change and Military Vehicles. Our goal is to learn to cat- egorize such image-text posts as pristine or fal- sified (out-of-context) by means of detecting se- mantic inconsistencies between images and text. To that end, we first collect a large-scale dataset of multimodal tweets, Twitter-COMMs, with ∗Denotes equal contribution. over 884k tweets. In our approach, we fuse input image and text embeddings generated by CLIP (Radford et al., 2021) via an elementwise product, and train a classifier to distinguish real tweets from automatically constructed random and hard mismatches. To validate this approach and demonstrate the usefulness of the Twitter-COMMs dataset, we report results on human-generated test data, created to mimic real-world misinfor- mation. We discuss the results and model abla- tions, and provide additional insights into the chal- lenges of this task. Our dataset is publicly avail- able at: https://github.com/GiscardBiamby/Twitter- COMMs. 2 Related Work There exist a number of large-scale Twitter datasets concentrated on topics such as COVID-19 (Banda et al., 2021) or Climate Change (Littman and Wrubel, 2019). However, it remains difficult to collect labeled misinformation. Researchers have collected COVID-19 misconceptions on social me- dia via manual annotation (Hossain et al., 2020) or by linking to fact checking articles …

Content truncated to 500 words