Topic-relevant Response Generation using Optimal Transport for an Open-domain Dialog System

Proceedings of the 28th International Conference on Computational Linguistics, pages 4067–4077 Barcelona, Spain (Online), December 8-13, 2020 4067 Topic-relevant Response Generation using Optimal Transport for an Open-domain Dialog System Shuying Zhang Kyoto University Tianyu Zhao Kyoto University {zhang,zhao,kawahara}@sap.ist.i.kyoto-u.ac.jp Tatsuya Kawahara Kyoto University Abstract Conventional neural generative models tend to generate safe and generic responses which have little connection with previous utterances semantically and would disengage users in a dialog system. To generate relevant responses, we propose a method that employs two types of con- straints - topical constraint and semantic constraint. Under the hypothesis that a response and its context have higher relevance when they share the same topics, the topical constraint encourages the topics of a response to match its context by conditioning response decoding on topic words’ embeddings. The semantic constraint, which encourages a response to be semantically related to its context by regularizing the decoding objective function with semantic distance, is proposed. Optimal transport is applied to compute a weighted semantic distance between the representation of a response and the context. Generated responses are evaluated by automatic metrics, as well as human judgment, showing that the proposed method can generate more topic-relevant and content-rich responses than conventional models. 1 Introduction In the past decade, personal assistants based on a speech dialog system have shown significant poten- tial and become a trend, which have aroused great interest in dialog systems. Recently, because of the access to large public conversation datasets, neural response generation has been investigated for an open-domain dialog system. Neural response generation is to learn a response generation model under sequence-to-sequence (seq2seq) (Bahdanau et al., 2014) framework. Such a model is trained to optimize the conditional likelihood P(Y |X) of generating a response Y given its context X. This objective func- tion is especially suitable for tasks like machine translation, of which the source and its target translation are semantically corresponding. However, for open-domain dialog response generation, a response can be weakly relevant to its previous utterances. There usually does not exist an obvious alignment between previous utterances and the response. Thus, neural generative models tend to generate safe and generic response, which contains little meaningful information, such as “really?”, and “I don’t know.”. These generic responses are usually short and vague, and can seldom provide effective information as feedback to the other partner in the dialog. This has been identified as one of major challenges in neural response generation (Li et al., 2016). To ameliorate the problem of generic responses, we design a model that can generate responses that are relevant to previous utterances. Two constraints are introduced, 1) topical constraint and 2) semantic constraint, to modify the objective function P(Y |X). Responses which are highly related to their con- text usually have the same topic as the context does. Thus, the topical constraint conditions the model’s decoding process on topic words in the context, which are identified by a topic word sequence labeler. Moreover, a relevant response should be semantically related to its context. …

Content truncated to 500 words