Gated Self-Matching Networks for Reading Comprehension and Question Answering

Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 189–198 Vancouver, Canada, July 30 - August 4, 2017. c⃝2017 Association for Computational Linguistics https://doi.org/10.18653/v1/P17-1018 Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 189–198 Vancouver, Canada, July 30 - August 4, 2017. c⃝2017 Association for Computational Linguistics https://doi.org/10.18653/v1/P17-1018 Gated Self-Matching Networks for Reading Comprehension and Question Answering Wenhui Wang†♮§∗ Nan Yang‡§ Furu Wei‡ Baobao Chang†♮ Ming Zhou‡ †Key Laboratory of Computational Linguistics, Peking University, MOE, China ‡Microsoft Research, Beijing, China ♮Collaborative Innovation Center for Language Ability, Xuzhou, 221009, China {wangwenhui,chbb}@pku.edu.cn {nanya,fuwei,mingzhou}@microsoft.com Abstract In this paper, we present the gated self- matching networks for reading compre- hension style question answering, which aims to answer questions from a given pas- sage. We first match the question and pas- sage with gated attention-based recurrent networks to obtain the question-aware pas- sage representation. Then we propose a self-matching attention mechanism to re- fine the representation by matching the passage against itself, which effectively encodes information from the whole pas- sage. We finally employ the pointer net- works to locate the positions of answers from the passages. We conduct extensive experiments on the SQuAD dataset. The single model achieves 71.3% on the evalu- ation metrics of exact match on the hidden test set, while the ensemble model further boosts the results to 75.9%. At the time of submission of the paper, our model holds the first place on the SQuAD leaderboard for both single and ensemble model. 1 Introduction In this paper, we focus on reading comprehension style question answering which aims to answer questions given a passage or document. We specif- ically focus on the Stanford Question Answering Dataset (SQuAD) (Rajpurkar et al., 2016), a large- scale dataset for reading comprehension and ques- tion answering which is manually created through crowdsourcing. SQuAD constrains answers to the space of all possible spans within the refer- ence passage, which is different from cloze-style reading comprehension datasets (Hermann et al., ∗Contribution during internship at Microsoft Research. §Equal contribution. 2015; Hill et al., 2016) in which answers are sin- gle words or entities. Moreover, SQuAD requires different forms of logical reasoning to infer the an- swer (Rajpurkar et al., 2016). Rapid progress has been made since the release of the SQuAD dataset. Wang and Jiang (2016b) build question-aware passage representation with match-LSTM (Wang and Jiang, 2016a), and pre- dict answer boundaries in the passage with pointer networks (Vinyals et al., 2015). Seo et al. (2016) introduce bi-directional attention flow networks to model question-passage pairs at multiple levels of granularity. Xiong et al. (2016) propose dynamic co-attention networks which attend the question and passage simultaneously and iteratively refine answer predictions. Lee et al. (2016) and Yu et al. (2016) predict answers by ranking continuous text spans within passages. Inspired by Wang and Jiang (2016b), we in- troduce a gated self-matching network, illustrated in Figure 1, an end-to-end neural network model for reading comprehension and question answer- ing. Our model consists of …

Content truncated to 500 words