Neural Responding Machine for Short-Text Conversation

Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, pages 1577–1586, Beijing, China, July 26-31, 2015. c⃝2015 Association for Computational Linguistics Neural Responding Machine for Short-Text Conversation Lifeng Shang, Zhengdong Lu, Hang Li Noah’s Ark Lab Huawei Technologies Co. Ltd. Sha Tin, Hong Kong {shang.lifeng,lu.zhengdong,hangli.hl}@huawei.com Abstract We propose Neural Responding Ma- chine (NRM), a neural network-based re- sponse generator for Short-Text Conver- sation. NRM takes the general encoder- decoder framework: it formalizes the gen- eration of response as a decoding process based on the latent representation of the in- put text, while both encoding and decod- ing are realized with recurrent neural net- works (RNN). The NRM is trained with a large amount of one-round conversation data collected from a microblogging ser- vice. Empirical study shows that NRM can generate grammatically correct and content-wise appropriate responses to over 75% of the input text, outperforming state- of-the-arts in the same setting, including retrieval-based and SMT-based models. 1 Introduction Natural language conversation is one of the most challenging artificial intelligence problems, which involves language understanding, reason- ing, and the utilization of common sense knowl- edge. Previous works in this direction mainly fo- cus on either rule-based or learning-based meth- ods (Williams and Young, 2007; Schatzmann et al., 2006; Misu et al., 2012; Litman et al., 2000). These types of methods often rely on manual effort in designing rules or automatic training of model with a particular learning algorithm and a small amount of data, which makes it difficult to develop an extensible open domain conversation system. Recently due to the explosive growth of mi- croblogging services such as Twitter1 and Weibo2, the amount of conversation data available on the web has tremendously increased. This makes a 1https://twitter.com/. 2http://www.weibo.com/. data-driven approach to attack the conversation problem (Ji et al., 2014; Ritter et al., 2011) pos- sible. Instead of multiple rounds of conversation, the task at hand, referred to as Short-Text Conver- sation (STC), only considers one round of conver- sation, in which each round is formed by two short texts, with the former being an input (referred to as post) from a user and the latter a response given by the computer. The research on STC may shed light on understanding the complicated mechanism of natural language conversation. Previous methods for STC fall into two cat- egories, 1) the retrieval-based method (Ji et al., 2014), and 2) the statistical machine translation (SMT) based method (Sordoni et al., 2015; Rit- ter et al., 2011). The basic idea of retrieval- based method is to pick a suitable response by ranking the candidate responses with a linear or non-linear combination of various matching fea- tures (e.g. number of shared words). The main drawbacks of the retrieval-based method are the following • the responses are pre-existing and hard to cus- tomize for the particular text or requirement from the task, e.g., style or attitude. • the use of matching features alone is usu- ally …

Content truncated to 500 words