World Knowledge for Reading Comprehension: Rare Entity Prediction with Hierarchical LSTMs Using External Descriptions

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 825–834 Copenhagen, Denmark, September 7–11, 2017. c⃝2017 Association for Computational Linguistics World Knowledge for Reading Comprehension: Rare Entity Prediction with Hierarchical LSTMs Using External Descriptions Teng Long, Emmanuel Bengio, Ryan Lowe Jackie Chi Kit Cheung, Doina Precup {teng.long,emmanuel.bengio,ryan.lowe}@mail.mcgill.ca {jcheung,dprecup}@cs.mcgill.ca School of Computer Science McGill University Abstract Humans interpret texts with respect to some background information, or world knowledge, and we would like to develop automatic reading comprehension systems that can do the same. In this paper, we in- troduce a task and several models to drive progress towards this goal. In particular, we propose the task of rare entity predic- tion: given a web document with several entities removed, models are tasked with predicting the correct missing entities con- ditioned on the document context and the lexical resources. This task is challenging due to the diversity of language styles and the extremely large number of rare enti- ties. We propose two recurrent neural net- work architectures which make use of ex- ternal knowledge in the form of entity de- scriptions. Our experiments show that our hierarchical LSTM model performs signif- icantly better at the rare entity prediction task than those that do not make use of ex- ternal resources. 1 Introduction Reading comprehension is the ability to process some text and understand its contents, in order to form some beliefs about the world. The starting point of this paper is the fact that world knowledge plays a crucial role in human reading comprehen- sion and language understanding. Work in the psy- chology of reading literature has demonstrated this point, for example by showing that readers are bet- ter able to recall the contents of a story when it de- scribes a counter-intuitive but plausible sequence of events, rather than a bizarre or a highly pre- dictable one (Barrett and Nyhof, 2001). This point is also central to work in the Schankian tradition of scripts (Schank and Abelson, 1977). Despite the importance of world knowledge, previous data sets and tasks for reading compre- hension have targeted other aspects of the read- ing comprehension problem, at times explicitly at- tempting to factor out its influence. In the Daily Mail/CNN dataset (Hermann et al., 2015), named entities such Clarkson and Top Gear are replaced by anonymized entity tokens like ent212. The Children’s Book Test focuses on the role of con- text and memory (Hill et al., 2016a), and the fic- tional genre makes it difficult to connect the enti- ties in the stories to real-world knowledge about those entities. As a result, language models have proved to be a highly competitive solution to these tasks. Chen et al. (2016) showed that their attention- based LSTM model achieves state-of-the-art re- sults on the Daily Mail/CNN data set. In fact, their analysis shows that more than half of the ques- tions can be answered by exact word matching and sentence-level paraphrase detection, and that many of the remaining errors are difficult to …

Content truncated to 500 words