Natural Answer Generation with Heterogeneous Memory

Proceedings of NAACL-HLT 2018, pages 185–195 New Orleans, Louisiana, June 1 - 6, 2018. c⃝2018 Association for Computational Linguistics Natural Answer Generation with Heterogeneous Memory Yao Fu and Yansong Feng Institute of Computer Science and Technology, Peking University The MOE Key Laboratory of Computational Linguistics, Peking University {francis_yao,fengyansong}@pku.edu.cn Abstract Memory augmented encoder-decoder frame- work has achieved promising progress for nat- ural language generation tasks. Such frame- works enable a decoder to retrieve from a memory during generation. However, less re- search has been done to take care of the mem- ory contents from different sources, which are often of heterogeneous formats. In this work, we propose a novel attention mechanism to en- courage the decoder to actively interact with the memory by taking its heterogeneity into account. Our solution attends across the gen- erated history and memory to explicitly avoid repetition, and introduce related knowledge to enrich our generated sentences. Experiments on the answer sentence generation task show that our method can effectively explore het- erogeneous memory to produce readable and meaningful answer sentences while maintain- ing high coverage for given answer informa- tion. 1 Introduction Most previous question answering systems fo- cus on finding candidate words, phrases or sen- tence snippets from many resources, and ranking them for their users (Chu-Carroll et al., 2004; Xu et al., 2016). Typically, candidate answers are col- lected from different resources, such as knowledge base (KB) or textual documents, which are often with heterogeneous formats, e.g., KB triples or semi-structured results from Information Extrac- tion (IE). For factoid questions, a single answer word or phrase is chosen as the response for users, as shown in Table 1 (A1). However, in many real-world scenarios, users may prefer more natural responses rather than a single word. For example, as A2 in Table 1, James Cameron directed the Titanic. is more favorable than the single name James Cameron. A straight- forward solution to compose an answer sentence is to build a template based model, where the answer Q Who is the director of the Titanic? A1 James Cameron A2 James Cameron directed the Titanic. A3 James Cameron directed it. A4 James Cameron directed it in 1999. Table 1: Answer sentences generated by different QA systems word James Cameron and topic word in the ques- tion the Titanic are filled into a pre-defined tem- plate (Chu-Carroll et al., 2004). But such systems intrinsically lack variety, hence hard to generalize to new domains. To produce more natural answer sentences, Yin et al. (2015) proposed GenQA, an encoder- decoder based model to select candidate answers from a KB styled memory during decoding to gen- erate an answer sentence. CoreQA (He et al., 2017b) further extended GenQA with a copy mechanism to learn to copy words from the ques- tion. The application of attention mechanism en- ables those attempts to successfully learn sentence varieties from the memory and training data, such as usage of pronouns (A3 in Table 1). However, since they are within the encoder-decoder frame- work, they also …

Content truncated to 500 words