Back to the Future: Bidirectional Information Decoupling Network for Multi-turn Dialogue Modeling

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 2761 - 2774 December 7-11, 2022 ©2022 Association for Computational Linguistics Back to the Future: Bidirectional Information Decoupling Network for Multi-turn Dialogue Modeling Yiyang Li1,2, Hai Zhao1,2,∗and Zhuosheng Zhang1,2 1 Department of Computer Science and Engineering, Shanghai Jiao Tong University 2 Key Laboratory of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering, Shanghai Jiao Tong University {eric-lee,zhangzs}@sjtu.edu.cn,zhaohai@cs.sjtu.edu.cn Abstract Multi-turn dialogue modeling as a challeng- ing branch of natural language understanding (NLU), aims to build representations for ma- chines to understand human dialogues, which provides a solid foundation for multiple down- stream tasks. Recent studies of dialogue mod- eling commonly employ pre-trained language models (PrLMs) to encode the dialogue history as successive tokens, which is insufficient in capturing the temporal characteristics of dia- logues. Therefore, we propose Bidirectional Information Decoupling Network (BiDeN) as a universal dialogue encoder, which explic- itly incorporates both the past and future con- texts and can be generalized to a wide range of dialogue-related tasks. Experimental re- sults on datasets of different downstream tasks demonstrate the universality and effectiveness of our BiDeN. The official implementation of BiDeN is available at https://github.com/ EricLee8/BiDeN. 1 Introduction Multi-turn dialogue modeling as one of the core tasks in natural language understanding, aims to build representations for machines to understand human dialogues. It is the foundation of solving multiple dialogue-related tasks such as selecting a response (Lowe et al., 2015; Zhang et al., 2018; Cui et al., 2020), answering questions (Sun et al., 2019a; Yang and Choi, 2019; Li et al., 2020a), or making a summarization according to the dialogue history (Gliwa et al., 2019; Chen et al., 2021). Dialogue contexts possess their intrinsic nature of informal, colloquial expressions, discontinu- ous semantics, and strong temporal characteristics (Reddy et al., 2019; Yang and Choi, 2019; Chen et al., 2020; Qin et al., 2021a), making them harder for machines to understand compared to plain texts ∗ Corresponding author. This paper was partially sup- ported by Key Projects of National Natural Science Founda- tion of China (U1836222 and 61733011). (Rajpurkar et al., 2016; Cui et al., 2020; Zhang et al., 2021). To tackle the aforementioned obsta- cles, most of the existing works on dialogue mod- eling have made efforts from three perspectives. The first group of works adopt a hierarchical en- coding strategy by first encoding each utterance in a dialogue separately, then making them interact with each other by an utterance-level interaction module (Zhang et al., 2018; Li and Choi, 2020; Gu et al., 2021). This strategy shows sub-optimal to model multi-turn dialogue owing to the neglect of informative dialogue contexts when encoding individual utterances. The second group of works simply concatenate all the utterances chronologi- cally as a whole (together with response candidates for the response selection task), then encode them using pre-trained language models (PrLMs) (Zhang et al., 2020a; Smith et al., 2020). This encoding pattern has its advantage of leveraging the strong interaction ability of self-attention layer …

Content truncated to 500 words