Learning and Knowledge Transfer with Memory Networks for Machine Comprehension

Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pages 850–859, Valencia, Spain, April 3-7, 2017. c⃝2017 Association for Computational Linguistics Learning and Knowledge Transfer with Memory Networks for Machine Comprehension Mohit Yadav TCS Research New-Delhi y.mohit@tcs.com Lovekesh Vig TCS Research New-Delhi lovekesh.vig@tcs.com Gautam Shroff TCS Research New-Delhi gautam.shroff@tcs.com Abstract Enabling machines to read and compre- hend unstructured text remains an unful- filled goal for NLP research. Recent re- search efforts on the “machine compre- hension” task have managed to achieve close to ideal performance on simulated data. However, achieving similar lev- els of performance on small real world datasets has proved difficult; major chal- lenges stem from the large vocabulary size, complex grammar, and the frequent ambiguities in linguistic structure. On the other hand, the requirement of human gen- erated annotations for training, in order to ensure a sufficiently diverse set of ques- tions is prohibitively expensive. Moti- vated by these practical issues, we propose a novel curriculum inspired training pro- cedure for Memory Networks to improve the performance for machine comprehen- sion with relatively small volumes of train- ing data. Additionally, we explore various training regimes for Memory Networks to allow knowledge transfer from a closely related domain having larger volumes of labelled data. We also suggest the use of a loss function to incorporate the asymmet- ric nature of knowledge transfer. Our ex- periments demonstrate improvements on Dailymail, CNN, and MCTest datasets. 1 Introduction A long-standing goal of NLP is to imbue machines with the ability to comprehend text and answer natural language questions. The goal is still dis- tant and yet generates tremendous amount of in- terest due to the large number of potential NLP applications that are currently stymied because of their inability to deal with unstructured text. Also, the next generation of search engines are aiming to provide precise and semantically relevant an- swers in response to questions-as-queries; similar to the functionality of digital assistants like Cor- tana and Siri. This will require text understanding at a non-superficial level, in addition to reasoning, and, making complex inferences about the text. As pointed out by Weston et al. (2016), the Question Answering (QA) task on unstructured text is a sound benchmark on which to evaluate machine comprehension. The authors also intro- duced bAbI: a simulation dataset for QA with mul- tiple toy tasks. These toy tasks require a machine to perform simple induction, deduction, multi- ple chaining of facts, and, complex reasoning; which make them a sound benchmark to measure progress towards AI-complete QA (Weston et al., 2016). The recently proposed Memory Network architecture and its variants have achieved close to ideal performance, i.e., more than 95% accuracy on 16 out of a total of 20 QA tasks (Sukhbaatar et al., 2015; Weston et al., 2016). While this performance is impressive, and is indicative of the memory network having suf- ficient capacity for the machine comprehension task, the performance does not translate to real world …

Content truncated to 500 words