Natural Language Comprehension with the EpiReader

Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 128–137, Austin, Texas, November 1-5, 2016. c⃝2016 Association for Computational Linguistics Natural Language Comprehension with the EpiReader Adam Trischler adam.trischler Zheng Ye jeff.ye Xingdi Yuan eric.yuan Philip Bachman phil.bachman Alessandro Sordoni alessandro.sordoni Kaheer Suleman k.suleman @maluuba.com Maluuba Research Montréal, Québec, Canada Abstract We present EpiReader, a novel model for ma- chine comprehension of text. Machine com- prehension of unstructured, real-world text is a major research goal for natural language pro- cessing. Current tests of machine comprehen- sion pose questions whose answers can be in- ferred from some supporting text, and evaluate a model’s response to the questions. EpiReader is an end-to-end neural model comprising two components: the first component proposes a small set of candidate answers after compar- ing a question to its supporting text, and the second component formulates hypotheses us- ing the proposed candidates and the question, then reranks the hypotheses based on their esti- mated concordance with the supporting text. We present experiments demonstrating that EpiReader sets a new state-of-the-art on the CNN and Children’s Book Test benchmarks, outperforming previous neural models by a sig- nificant margin. 1 Introduction When humans reason about the world, we tend to for- mulate a variety of hypotheses and counterfactuals, then test them in turn by physical or thought exper- iments. The philosopher Epicurus first formalized this idea in his Principle of Multiple Explanations: if several theories are consistent with the observed data, retain them all until more data is observed. In this pa- per, we argue that the same principle can be applied to machine comprehension of natural language. We propose a deep neural comprehension model, trained end-to-end, that we call EpiReader. Comprehension of natural language by machines, at a near-human level, is a prerequisite for an ex- tremely broad class of useful applications of artificial intelligence. Indeed, most human knowledge is col- lected in the natural language of text. Machine com- prehension (MC) has therefore garnered significant attention from the machine learning research commu- nity. Machine comprehension is typically evaluated by posing a set of questions based on a supporting text passage, then scoring a system’s answers to those questions. Such tests are objectively gradable and may assess a range of abilities, from basic understand- ing to causal reasoning to inference (Richardson et al., 2013). In the past year, two large-scale MC datasets have been released: the CNN/Daily Mail corpus, consist- ing of news articles from those outlets (Hermann et al., 2015), and the Children’s Book Test (CBT), consisting of short excerpts from books available through Project Gutenberg (Hill et al., 2016). The size of these datasets (on the order of 105 distinct questions) makes them amenable to data-intensive deep learning techniques. Both corpora use Cloze- style questions (Taylor, 1953), which are formulated by replacing a word or phrase in a given sentence with a placeholder token. The task is then to find the answer that “fills in the blank”. In tandem with these corpora, a …

Content truncated to 500 words