Text Understanding with the Attention Sum Reader Network

Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pages 908–918, Berlin, Germany, August 7-12, 2016. c⃝2016 Association for Computational Linguistics Text Understanding with the Attention Sum Reader Network Rudolf Kadlec, Martin Schmid, Ondrej Bajgar & Jan Kleindienst IBM Watson V Parku 4, Prague, Czech Republic {rudolf kadlec,martin.schmid,obajgar,jankle}@cz.ibm.com Abstract Several large cloze-style context-question- answer datasets have been introduced re- cently: the CNN and Daily Mail news data and the Children’s Book Test. Thanks to the size of these datasets, the asso- ciated text comprehension task is well suited for deep-learning techniques that currently seem to outperform all alterna- tive approaches. We present a new, simple model that uses attention to directly pick the answer from the context as opposed to computing the answer using a blended rep- resentation of words in the document as is usual in similar models. This makes the model particularly suitable for question- answering problems where the answer is a single word from the document. Ensem- ble of our models sets new state of the art on all evaluated datasets. 1 Introduction Most of the information humanity has gathered up to this point is stored in the form of plain text. Hence the task of teaching machines how to un- derstand this data is of utmost importance in the field of Artificial Intelligence. One way of testing the level of text understanding is simply to ask the system questions for which the answer can be in- ferred from the text. A well-known example of a system that could make use of a huge collection of unstructured documents to answer questions is for instance IBM’s Watson system used for the Jeop- ardy challenge (Ferrucci et al., 2010). Cloze style questions (Taylor, 1953), i.e. ques- tions formed by removing a phrase from a sen- tence, are an appealing form of such questions (for example see Figure 1). While the task is easy to evaluate, one can vary the context, the question Document: What was supposed to be a fantasy sports car ride at Walt Disney World Speedway turned deadly when a Lamborghini crashed into a guardrail. The crash took place Sunday at the Exotic Driving Experi- ence, which bills itself as a chance to drive your dream car on a racetrack. The Lamborghini’s passenger, 36- year-old Gary Terry of Davenport, Florida, died at the scene, Florida Highway Patrol said. The driver of the Lamborghini, 24-year-old Tavon Watson of Kissimmee, Florida, lost control of the vehicle, the Highway Patrol said. (...) Question: Officials say the driver, 24-year-old Tavon Watson, lost control of a Answer candidates: Tavon Watson, Walt Disney World Speedway, Highway Patrol, Lamborghini, Florida, (...) Answer: Lamborghini Figure 1: Each example consists of a context document, question, answer cadidates and, in the training data, the correct answer. This example was taken from the CNN dataset (Hermann et al., 2015). Anonymization of this example that makes the task harder is shown in Table 3. sentence or the specific phrase missing in the ques- tion to dramatically …

Content truncated to 500 words