Globally Normalized Reader

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1059–1069 Copenhagen, Denmark, September 7–11, 2017. c⃝2017 Association for Computational Linguistics Globally Normalized Reader Jonathan Raiman and John Miller Baidu Silicon Valley Artificial Intelligence Lab {jonathanraiman,millerjohn}@baidu.com Abstract Rapid progress has been made towards question answering (QA) systems that can extract answers from text. Existing neu- ral approaches make use of expensive bi- directional attention mechanisms or score all possible answer spans, limiting scala- bility. We propose instead to cast extrac- tive QA as an iterative search problem: select the answer’s sentence, start word, and end word. This representation re- duces the space of each search step and al- lows computation to be conditionally allo- cated to promising search paths. We show that globally normalizing the decision pro- cess and back-propagating through beam search makes this representation viable and learning efficient. We empirically demonstrate the benefits of this approach using our model, Globally Normalized Reader (GNR), which achieves the sec- ond highest single model performance on the Stanford Question Answering Dataset (68.4 EM, 76.21 F1 dev) and is 24.7x faster than bi-attention-flow. We also in- troduce a data-augmentation method to produce semantically valid examples by aligning named entities to a knowledge base and swapping them with new entities of the same type. This method improves the performance of all models considered in this work and is of independent interest for a variety of NLP tasks. 1 Introduction Question answering (QA) and information extrac- tion systems have proven to be invaluable in wide variety of applications such as medical informa- tion collection on drugs and genes (Quirk and Who was first to recognize that the Analytical Engine had applications beyond pure calculation? Ada Lovelace was known for her work on Charles Babbage's Analytical Engine. She was the first to recognize that the machine had applications beyond calculation. Ada Lovelace was known for her work on Charles Babbage's Analytical Engine. Lovelace was known for her work on Charles Babbage's Analytical Engine. Charles Babbage's Analytical Engine. Charles Babbage’s Analytical Engine Charles Babbage Ada Lovelace Span Stop Span Start Sentence 0.51 0.49 0.55 0.09 0.36 0.64 0.16 0.20 Decision Boundary 0.9 State probability Prediction Decision Word pick Question Figure 1: GNR answering a question. It first picks a sentence, then start word, then end word. Probabilities are global and normalized over the beam. Model initially picks the wrong sentence, but global normalization lets it recover. Final prediction’s probability (0.64) exceeds sentence pick (0.49), whereas with local normalization each probability is upper bounded by the previous step. Poon, 2016), large scale health impact studies (Al- thoff et al., 2016), or educational material develop- ment (Koedinger et al., 2015). Recent progress in neural-network based extractive question answer- ing models are quickly closing the gap with human performance on several benchmark QA tasks such as SQuAD (Rajpurkar et al., 2016), MS MARCO (Nguyen et al., 2016), or NewsQA (Trischler et al., 2016a). However, current approaches to extractive question answering face several limitations: 1059 1. Computation …

Content truncated to 500 words