Diversifying Dialogue Generation with Non-Conversational Text

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7087–7097 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 7087 Diversifying Dialogue Generation with Non-Conversational Text Hui Su1∗, Xiaoyu Shen2∗ Sanqiang Zhao3, Xiao Zhou1, Pengwei Hu4, Randy Zhong1, Cheng Niu1 and Jie Zhou1 1Pattern Recognition Center, Wechat AI, Tencent Inc, China 2MPI Informatics & Spoken Language Systems (LSV), Saarland Informatics Campus 3University of Pittsburgh 4The Hong Kong Polytechnic University, Hong Kong aaronsu@tencent.com,xshen@mpi-inf.mpg.de Abstract Neural network-based sequence-to-sequence (seq2seq) models strongly suffer from the low- diversity problem when it comes to open- domain dialogue generation. As bland and generic utterances usually dominate the fre- quency distribution in our daily chitchat, avoid- ing them to generate more interesting re- sponses requires complex data filtering, sam- pling techniques or modifying the training objective. In this paper, we propose a new perspective to diversify dialogue gener- ation by leveraging non-conversational text. Compared with bilateral conversations, non- conversational text are easier to obtain, more diverse and cover a much broader range of topics. We collect a large-scale non- conversational corpus from multi sources in- cluding forum comments, idioms and book snippets. We further present a training paradigm to effectively incorporate these text via iterative back translation. The resulting model is tested on two conversational datasets and is shown to produce significantly more diverse responses without sacrificing the rele- vance with context. 1 Introduction Seq2seq models have achieved impressive success in a wide range of text generation tasks. In open- domain chitchat, however, people have found the model tends to strongly favor short, generic re- sponses like “I don’t know” or “OK” (Vinyals and Le, 2015; Shen et al., 2017a). The reason lies in the extreme one-to-many mapping relation between ev- ery context and its potential responses (Zhao et al., 2017; Su et al., 2018). Generic utterances, which can be in theory paired with most context, usually dominate the frequency distribution in the dialogue training corpus and thereby pushes the model to ∗Equal contribution. Conversational Text Context 暗恋的人却不喜欢我 (Translation) The one I have a crush on doesn’t like me. Response 摸摸头 Head pat. Non-Conversational Text Forum Comments 暗恋这碗酒，谁喝都会醉啊 Crush is an alcoholic drink, whoever drinks it will get intoxicated. Idiom 何必等待一个没有结果的等待 Why wait for a result without hope Book Snippet 真诚的爱情之路永不会是平坦的 The course of true love never did run smooth (From A Midsummer Night’s Dream) Table 1: A daily dialogue and non-conversational text from three sources. The contents of non-conversational text can be potentially utilized to enrich the response generation. blindly produce these safe, dull responses (Su et al., 2019b; Cs´aky et al., 2019) Current solutions can be roughly categorized into two classes: (1) Modify the seq2seq itself to bias toward diverse responses (Li et al., 2016a; Shen et al., 2019a). However, the model is still trained on the limited dialogue corpus which re- stricts its power at covering broad topics in open- domain chitchat. (2) Augment the training corpus with extra information like structured world knowl- edge, personality or emotions (Li …

Content truncated to 500 words