Multimodal Fusion with Co-Attention Networks for Fake News Detection

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 2560–2569 August 1–6, 2021. ©2021 Association for Computational Linguistics 2560 Multimodal Fusion with Co-Attention Networks for Fake News Detection Yang Wu 1,2, Pengwei Zhan1,2, Yunjian Zhang1,2, Liming Wang1∗, Zhen Xu1 1Institute of Information Engineering, Chinese Academy of Sciences 2School of Cyber Security, University of Chinese Academy of Sciences {wuyang0419,zhanpengwei,zhangyunjian,wangliming,xuzhen}@iie.ac.cn Abstract Fake news with textual and visual contents has a better story-telling ability than text-only con- tents, and can be spread quickly with social media. People can be easily deceived by such fake news, and traditional expert identification is labor-intensive. Therefore, automatic detec- tion of multimodal fake news has become a new hot-spot issue. A shortcoming of exist- ing approaches is their inability to fuse multi- modality features effectively. They simply concatenate unimodal features without consid- ering inter-modality relations. Inspired by the way people read news with image and text, we propose a novel Multimodal Co-Attention Networks (MCAN) to better fuse textual and visual features for fake news detection. Ex- tensive experiments conducted on two real- world datasets demonstrate that MCAN can learn inter-dependencies among multimodal features and outperforms state-of-the-art meth- ods. 1 Introduction The rapid growth of social media has created fer- tile soil for the emergence and fast spread of fake news (Zhao et al., 2015), resulting in serious conse- quences. For example, during the U.S. 2016 pres- idential election, the most popular fake news was more widely spread than the most popular authen- tic news on Facebook, which confused people and broke the authenticity balance of the news ecosys- tem (Shu et al., 2017). To mitigate the negative effects caused by fake news, it is crucial to detect fake news on social media automatically. Tweets with images are getting popular on social media recently, which have richer information and attract more viewers than tweets with only texts (Jin et al., 2017). Fake news also makes full use of this advantage to draw and mislead readers. Figure 1 shows three examples of fake news from Twitter. ∗Corresponding Author. Sharks in the mall! After the hurricane sandy! Lenticular Clouds over Mount Fuji. Woman, 36, gives birth to 14 children from 14 different fathers. Figure 1: Some fake news from Twitter. In the left example, both text and image indicate it is likely to be fake. The text of the middle one provides little evidence that it is fake news, but the image is obviously forged. In the right example, the image seems normal, while the textual contents in- dicate that it is probably fake. A hypothesis drawn from these examples is that combining text and the attached image is more conducive to detecting fake news. Recent works have a growing interest in using multimodal (text + image) information to detect fake news. Jin et al.(2017) utilize local attention mechanisms to fuse features of image, text, and social context. Some studies explore to learn the joint representations of text and image, based on auxiliary adversarial networks (Wang et al., 2018) and …

Content truncated to 500 words