Dynamic Entity Representation with Max-pooling Improves Machine Reading

Proceedings of NAACL-HLT 2016, pages 850–855, San Diego, California, June 12-17, 2016. c⃝2016 Association for Computational Linguistics Dynamic Entity Representation with Max-pooling Improves Machine Reading Sosuke Kobayashi and Ran Tian and Naoaki Okazaki and Kentaro Inui Tohoku University, Japan {sosuke.k, tianran, okazaki, inui}@ecei.tohoku.ac.jp Abstract We propose a novel neural network model for machine reading, DER Network, which ex- plicitly implements a reader building dynamic meaning representations for entities by gath- ering and accumulating information around the entities as it reads a document. Eval- uated on a recent large scale dataset (Her- mann et al., 2015), our model exhibits bet- ter results than previous research, and we find that max-pooling is suited for model- ing the accumulation of information on enti- ties. Further analysis suggests that our model can put together multiple pieces of informa- tion encoded in different sentences to an- swer complicated questions. Our code for the model is available at https://github. com/soskek/der-network 1 Introduction Machine reading systems (Poon et al., 2010; Richardson et al., 2013) can be tested on their ability to answer queries about contents of doc- uments that they read, thus a central problem is how the information of documents should be orga- nized in the system and retrieved by the queries. Recently, large scale datasets of document-query- answer triples have been constructed from online newspaper articles and their summaries (Hermann et al., 2015), by replacing named entities in the summaries with placeholders to form Cloze (Tay- lor, 1953) style questions (Figure 1). These datasets have enabled training and testing of complicated neural network models of hypothesized machine readers (Hermann et al., 2015; Hill et al., 2015). ( @entity1 ) @entity0 may be @entity2 in the popular @entity4 superhero films , but he recently dealt in some advanced bionic technology himself . @entity0 recently presented a robotic arm to young @entity7 , a @entity8 boy who is missing his right arm from just above his elbow . the arm was made by @entity12 , a …! " [X] " star @entity0 presents a young child with a bionic arm! Query ! Context ! (CNN)Robert Downey Jr. may be Iron Man in the popular Marvel superhero films, but he recently dealt in some advanced bionic technology himself. Downey recently presented a robotic arm to young Alex Pring, a Central Florida boy who is missing his right arm from just above his elbow. The arm was made by Limbitless Solutions, a …! "Iron Man" star Robert Downey Jr. presents a young child with a bionic arm! Raw Highlight ! Raw Article ! Answer @entity2! Figure 1: A document-query-answer triple con- structed from a news article and its bullet point sum- mary. An entity in the summary (Robert Downey Jr.) is replaced by the placeholder [X] to form a query. All entities are anonymized to exclude world knowl- edge and focus on reading comprehension. In this paper, we hypothesize that a reader without world knowledge can only understand a named en- tity by dynamically constructing its meaning from the …

Content truncated to 500 words