Enhancing Key-Value Memory Neural Networks for Knowledge Based Question Answering

Proceedings of NAACL-HLT 2019, pages 2937–2947 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 2937 Enhancing Key-Value Memory Neural Networks for Knowledge Based Question Answering Kun Xu1, Yuxuan Lai2, Yansong Feng2,3, Zhiguo Wang4 1Tencent AI Lab 2ICST, Peking University 3The MOE Key Laboratory of Computational Linguistics, Peking University 4Amazon syxu828@gmail.com {erutan,fengyansong}@pku.edu.cn zgw.tomorrow@gmail.com Abstract Traditional Key-value Memory Neural Net- works (KV-MemNNs) are proved to be effec- tive to support shallow reasoning over a col- lection of documents in domain specific Ques- tion Answering or Reading Comprehension tasks. However, extending KV-MemNNs to Knowledge Based Question Answering (KB- QA) is not trivia, which should properly de- compose a complex question into a sequence of queries against the memory, and update the query representations to support multi-hop reasoning over the memory. In this paper, we propose a novel mechanism to enable con- ventional KV-MemNNs models to perform in- terpretable reasoning for complex questions. To achieve this, we design a new query up- dating strategy to mask previously-addressed memory information from the query repre- sentations, and introduce a novel STOP strat- egy to avoid invalid or repeated memory read- ing without strong annotation signals. This also enables KV-MemNNs to produce struc- tured queries and work in a semantic pars- ing fashion. Experimental results on bench- mark datasets show that our solution, trained with question-answer pairs only, can provide conventional KV-MemNNs models with better reasoning abilities on complex questions, and achieve state-of-art performances. 1 Introduction Memory Neural Networks (MemNNs) [Weston et al., 2014; Sukhbaatar et al., 2015b] are a fam- ily of neural network models that aim to learn how to reason with a long-term memory com- ponent and various inference components. The memory component serves as a knowledge base to recall facts from the past. MemNNs have been successfully applied in many natural lan- guage processing applications such as question question text Embedding Key Addressing query representation Hashing k1 k2 k3 … kN v1 v2 v3 … vN memory slots Value Reading query update Ranking answer representation Embedding candidate answer representation answer text answer candidates + Figure 1: The Key-Value Memory Neural Network Ar- chitecture. answering and reading comprehension (RC). Re- cently, Miller et al. [2016] proposed a variant of MemNNs, namely Key-Value Memory Neural Networks (KV-MemNNs), which generalizes the original MemNNs by storing facts in a key-value structured memory. Figure 1 illustrates the basic architecture of KV-MemNNs, which consists of five components. The question is first fed to the Embedding component and the Hashing compo- nent. The former converts the incoming question to an internal feature representation. The Hashing component uses the question to pre-select a list of facts to compose the key-value memory. The Key Addressing component takes the input question representation and the current memory to compute the relevance probability between the question and each key in the memory. The Value Reading com- ponent reads the values of all addressed memories by taking their weighted sum using the relevance probabilities. The obtained value representation is then added …

Content truncated to 500 words