A Parallel-Hierarchical Model for Machine Comprehension on Sparse Data

Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pages 432–441, Berlin, Germany, August 7-12, 2016. c⃝2016 Association for Computational Linguistics A Parallel-Hierarchical Model for Machine Comprehension on Sparse Data Adam Trischler∗ adam.trischler Zheng Ye∗ jeff.ye Xingdi Yuan eric.yuan Jing He jing.he Philip Bachman phil.bachman Maluuba Research Montreal, Qu´ebec, Canada Kaheer Suleman k.suleman@maluuba.com Abstract Understanding unstructured text is a ma- jor goal within natural language process- ing. Comprehension tests pose questions based on short text passages to evaluate such understanding. In this work, we in- vestigate machine comprehension on the challenging MCTest benchmark. Partly because of its limited size, prior work on MCTest has focused mainly on engi- neering better features. We tackle the dataset with a neural approach, harness- ing simple neural networks arranged in a parallel hierarchy. The parallel hierarchy enables our model to compare the pas- sage, question, and answer from a vari- ety of trainable perspectives, as opposed to using a manually designed, rigid fea- ture set. Perspectives range from the word level to sentence fragments to sequences of sentences; the networks operate only on word-embedding representations of text. When trained with a methodology de- signed to help cope with limited training data, our Parallel-Hierarchical model sets a new state of the art for MCTest, outper- forming previous feature-engineered ap- proaches slightly and previous neural ap- proaches by a significant margin (over 15 percentage points). 1 Introduction Humans learn in a variety of ways—by communi- cation with each other and by study, the reading of text. Comprehension of unstructured text by machines, at a near-human level, is a major goal for natural language processing. It has garnered ∗A. Trischler and Z. Ye contributed equally to this work. significant attention from the machine learning re- search community in recent years. Machine comprehension (MC) is evaluated by posing a set of questions based on a text pas- sage (akin to the reading tests we all took in school). Such tests are objectively gradable and can be used to assess a range of abilities, from basic understanding to causal reasoning to infer- ence (Richardson et al., 2013). Given a text pas- sage and a question about its content, a system is tested on its ability to determine the correct an- swer (Sachan et al., 2015). In this work, we focus on MCTest, a complex but data-limited compre- hension benchmark, whose multiple-choice ques- tions require not only extraction but also infer- ence and limited reasoning (Richardson et al., 2013). Inference and reasoning are important hu- man skills that apply broadly, beyond language. We present a parallel-hierarchical approach to machine comprehension designed to work well in a data-limited setting. There are many use-cases in which comprehension over limited data would be handy: for example, user manuals, internal doc- umentation, legal contracts, and so on. More- over, work towards more efficient learning from any quantity of data is important in its own right, for bringing machines more in line with the way humans learn. Typically, artificial neural networks require …

Content truncated to 500 words