DialogConv: A Lightweight Fully Convolutional Network for Multi-view Response Selection

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 12086 - 12098 December 7-11, 2022 ©2022 Association for Computational Linguistics DialogConv: A Lightweight Fully Convolutional Network for Multi-view Response Selection Yongkang Liu1, Shi Feng1, Wei Gao2, Daling Wang1 and Yifei Zhang1 1 Northeastern University, China 2 Singapore Management University, Singapore misonsky@163.com, fengshi@cse.neu.edu.cn weigao@smu.edu.sg, {wangdaling,zhangyifei}@cse.neu.edu.cn Abstract Current end-to-end retrieval-based dialogue systems are mainly based on Recurrent Neural Networks or Transformers with attention mech- anisms. Although promising results have been achieved, these models often suffer from slow inference or huge number of parameters. In this paper, we propose a novel lightweight fully con- volutional architecture, called DialogConv, for response selection. DialogConv is exclusively built on top of convolution to extract matching features of context and response. Dialogues are modeled in 3D views, where DialogConv performs convolution operations on embedding view, word view and utterance view to capture richer semantic information from multiple con- textual views. On the four benchmark datasets, compared with state-of-the-art baselines, Di- alogConv is on average about 8.5× smaller in size, and 79.39× and 10.64× faster on CPU and GPU devices, respectively. At the same time, DialogConv achieves the competitive ef- fectiveness of response selection. 1 Introduction An important challenge in building intelligent di- alogue systems is the response selection problem, which aims to select an appropriate response from a set of candidates given a dialogue context. Such retrieval-based dialogue systems have attracted great attention from academia and industry due to the advantages of informative and fluent responses produced (Tao et al., 2021). The existing retrieval-based dialogue systems can be divided into three patterns according to the way of input handling (Zhang and Zhao, 2021): (i) Separate Pattern (Wu et al., 2017; Zhang et al., 2018; Zhou et al., 2018; Gu et al., 2019); (ii) Con- catenated Pattern (Tan et al., 2015; Zhou et al., 2016); (iii) PrLM (Pretrained Language Model) Pattern (Cui et al., 2020; Gu et al., 2020; Liu et al., 2021). Separate Pattern (i.e., Figure 1 (a)) encodes utterances individually, while Concatenated Pat- Figure 1: Flat modeling. (a) is separate pattern, (b) is concatenated pattern and (c) is PrLM pattern. Grey bars in (c) are embedded representations of special symbols. tern (i.e., Figure 1 (b)) concatenates all utterances into a continuous word sequence. Methods based on these two patterns usually have Recurrent Neu- ral Networks (RNNs) (Hochreiter and Schmidhu- ber, 1997; Cho et al., 2014) and attention mech- anism (Bahdanau et al., 2015) as the backbone. Although promising results have been achieved, these methods are generally slow in training and inference due to their recurrent nature. The PrLM Pattern (i.e., Figure 1 (c)) uses special symbols to connect all utterances into a continuous sequence, similar to Concatenated Pattern. While PrLM Pattern has obtained state-of-the-art perfor- mance in response selection (Cui et al., 2020; Gu et al., 2020; Liu et al., 2021), this method having Transformer (Vaswani et al., 2017) as the de facto standard architecture suffer from a large number of parameters …

Content truncated to 500 words