Tackling Fake News Detection by Continually Improving Social Context Representations using Graph Neural Networks

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 103 - 114 May 22-27, 2022 c⃝2022 Association for Computational Linguistics UniTranSeR: A Unified Transformer Semantic Representation Framework for Multimodal Task-Oriented Dialog Systems Zhiyuan Ma1, Jianjun Li1∗, Guohui Li1, Yongjing Cheng2 1 Huazhong University of Science and Technology (HUST), China 2 National University of Defense Technology (NUDT), China {zhiyuanma,jianjunli,guohuili}@hust.edu.cn davidcheng1001@163.com Abstract As a more natural and intelligent interac- tion manner, multimodal task-oriented dia- log system recently has received great atten- tion and many remarkable progresses have been achieved. Nevertheless, almost all ex- isting studies follow the pipeline to first learn intra-modal features separately and then conduct simple feature concatenation or attention-based feature fusion to generate re- sponses, which hampers them from learning inter-modal interactions and conducting cross- modal feature alignment for generating more intention-aware responses. To address these issues, we propose UniTranSeR, a Unified Transformer Semantic Representation frame- work with feature alignment and intention rea- soning for multimodal dialog systems. Specif- ically, we first embed the multimodal features into a unified Transformer semantic space to prompt inter-modal interactions, and then de- vise a feature alignment and intention reason- ing (FAIR) layer to perform cross-modal en- tity alignment and fine-grained key-value rea- soning, so as to effectively identify user’s in- tention for generating more accurate responses. Experimental results verify the effectiveness of UniTranSeR, showing that it significantly outperforms state-of-the-art approaches on the representative MMD dataset. 1 Introduction The multimodal task-oriented dialog systems are designed to help users achieve specific goals such as clothing recommendation or restaurant reserva- tion, which is in growing demand in the current business environment. As a leading study, Saha et al. (2018) released a multimodal dialog dataset (MMD) in the online retail domain. Based on such a benchmark dataset, many multimodal dialog mod- els incorporating domain knowledge have recently been proposed (Chauhan et al., 2019; Zhang et al., ∗Corresponding author. Attributes Value NAME jacket MATERIAL leather COLOR black IMAGE ... ... jacket jeans sling bag necklace platform heels Passion for jacket celebrities Celebrities Histogram Style tips Graph Attributes Table Multimodal Knowledge Base Show some similar jackets in black color. Hi Hello, how can i help you? Found some similar black leather-jackets for you. I like the 2nd one, will it go well with jeans? Is it popular among celebrities? Yes, it is a good match, and it's universally popular among celebrities. Multimodal Dialogue System Figure 1: Example of multimodal task-oriented dialog including multimodal entity alignment and knowledge query from the MMD dataset (Saha et al., 2018). Note that red marks the entities to be queried in the mul- timodal knowledge base and blue marks the acquired knowledge information. 2019, 2021), which basically exploit taxonomy- based method (Liao et al., 2018; Cui et al., 2019) or attention-based method (Nie et al., 2019; He et al., 2020) to incorporate knowledge base (KB) information for better performance. Though achieving remarkable progress, existing multimodal task-oriented dialog systems still suf- fer from the following three …

Content truncated to 500 words