A Neural Network Approach for Knowledge-Driven Response Generation

Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 3370–3380, Osaka, Japan, December 11-17 2016. A Neural Network Approach for Knowledge-Driven Response Generation Pavlos Vougiouklis Jonathon Hare Electronics and Computer Science University of Southampton Southampton, UK {pv1e13, jsh2, e.simperl}@ecs.soton.ac.uk Elena Simperl Abstract We present a novel response generation system. The system assumes the hypothesis that par- ticipants in a conversation base their response not only on previous dialog utterances but also on their background knowledge. Our model is based on a Recurrent Neural Network (RNN) that is trained over concatenated sequences of comments, a Convolution Neural Network that is trained over Wikipedia sentences and a formulation that couples the two trained embeddings in a multimodal space. We create a dataset of aligned Wikipedia sentences and sequences of Reddit utterances, which we we use to train our model. Given a sequence of past utterances and a set of sentences that represent the background knowledge, our end-to-end learnable model is able to generate context-sensitive and knowledge-driven responses by leveraging the alignment of two different data sources. Our approach achieves up to 55% improvement in perplexity compared to purely sequential models based on RNNs that are trained only on sequences of utterances. 1 Introduction Over the recent years, the level of users’ engagement and participation in public conversations on social media, such as Twitter, Facebook and Reddit has substantially increased. As a result, we now have large amounts of conversation data that can be used to train computer programs to be proficient conversation participants. Automatic response generation could be immediately deployable in social media as an auto- complete response suggestion feature or a conversation stimulant that adjusts the participation interest in a dialogue thread (Ritter et al., 2011). It should also be beneficial in the development of Question- Answering systems, by enhancing their ability to generate human-like responses (Grishman, 1979). Recent work on neural networks approaches shows their great potential at tackling a wide variety of Natural Language Processing (NLP) tasks (Bengio et al., 2003; Mikolov et al., 2010). Since a conver- sation can be perceived as a sequence of utterances, recent systems that are employed in the automatic response generation domain are based on Recurrent Neural Networks (RNNs) (Sordoni et al., 2015; Shang et al., 2015; Vinyals and Le, 2015), which are powerful sequence models. These systems base their generated response explicitly on a sequence of the most recent utterances of a conversation thread. Consequently, the sequence of characters, words, or comments, in a conversation, depending on the level of the model, is the only means with which these models achieve contextual-awareness, and in open-domain, realistic, situations it often proves inadequate (Vinyals and Le, 2015). In this paper we address the challenge of context-sensitive response generation. We build a dataset that aligns knowledge from Wikipedia in the form of sentences with sequences of Reddit utterances. The dataset consists of sequences of comments and a number of Wikipedia sentences that were allocated randomly from the Wikipedia pages to …

Content truncated to 500 words