Key-Value Memory Networks for Directly Reading Documents

Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1400–1409, Austin, Texas, November 1-5, 2016. c⃝2016 Association for Computational Linguistics Key-Value Memory Networks for Directly Reading Documents Alexander H. Miller1 Adam Fisch1 Jesse Dodge1,2 Amir-Hossein Karimi1 Antoine Bordes1 Jason Weston1 1Facebook AI Research, 770 Broadway, New York, NY, USA 2Language Technologies Institute, Carnegie Mellon University, Pittsburgh, PA, USA {ahm,afisch,jessedodge,ahkarimi,abordes,jase}@fb.com Abstract Directly reading documents and being able to answer questions from them is an unsolved challenge. To avoid its inherent difficulty, ques- tion answering (QA) has been directed towards using Knowledge Bases (KBs) instead, which has proven effective. Unfortunately KBs often suffer from being too restrictive, as the schema cannot support certain types of answers, and too sparse, e.g. Wikipedia contains much more information than Freebase. In this work we introduce a new method, Key-Value Memory Networks, that makes reading documents more viable by utilizing different encodings in the ad- dressing and output stages of the memory read operation. To compare using KBs, information extraction or Wikipedia documents directly in a single framework we construct an analysis tool, WIKIMOVIES, a QA dataset that contains raw text alongside a preprocessed KB, in the domain of movies. Our method reduces the gap between all three settings. It also achieves state-of-the-art results on the existing WIKIQA benchmark. 1 Introduction Question answering (QA) has been a long stand- ing research problem in natural language processing, with the first systems attempting to answer questions by directly reading documents (Voorhees and Tice, 2000). The development of large-scale Knowledge Bases (KBs) such as Freebase (Bollacker et al., 2008) helped organize information into structured forms, prompting recent progress to focus on answering questions by converting them into logical forms that can be used to query such databases (Berant et al., 2013; Kwiatkowski et al., 2013; Fader et al., 2014). Unfortunately, KBs have intrinsic limitations such as their inevitable incompleteness and fixed schemas that cannot support all varieties of answers. Since information extraction (IE) (Craven et al., 2000), in- tended to fill in missing information in KBs, is neither accurate nor reliable enough, collections of raw tex- tual resources and documents such as Wikipedia will always contain more information. As a result, even if KBs can be satisfactory for closed-domain problems, they are unlikely to scale up to answer general ques- tions on any topic. Starting from this observation, in this work we study the problem of answering by directly reading documents. Retrieving answers directly from text is harder than from KBs because information is far less struc- tured, is indirectly and ambiguously expressed, and is usually scattered across multiple documents. This explains why using a satisfactory KB—typically only available in closed domains—is preferred over raw text. We postulate that before trying to provide an- swers that are not in KBs, document-based QA sys- tems should first reach KB-based systems’ perfor- mance in such closed domains, where clear compari- son and evaluation is possible. To this end, this paper introduces WIKIMOVIES, a new analysis tool that …

Content truncated to 500 words