Explicit Utilization of General Knowledge in Machine Reading Comprehension

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2263–2272 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 2263 Explicit Utilization of General Knowledge in Machine Reading Comprehension Chao Wang and Hui Jiang Department of Electrical Engineering and Computer Science Lassonde School of Engineering, York University 4700 Keele Street, Toronto, Ontario, Canada {chwang, hj}@eecs.yorku.ca Abstract To bridge the gap between Machine Reading Comprehension (MRC) models and human beings, which is mainly reflected in the hunger for data and the robustness to noise, in this paper, we explore how to integrate the neu- ral networks of MRC models with the general knowledge of human beings. On the one hand, we propose a data enrichment method, which uses WordNet to extract inter-word semantic connections as general knowledge from each given passage-question pair. On the other hand, we propose an end-to-end MRC model named as Knowledge Aided Reader (KAR), which explicitly uses the above extracted gen- eral knowledge to assist its attention mecha- nisms. Based on the data enrichment method, KAR is comparable in performance with the state-of-the-art MRC models, and significantly more robust to noise than them. When only a subset (20%–80%) of the training examples are available, KAR outperforms the state-of- the-art MRC models by a large margin, and is still reasonably robust to noise. 1 Introduction Machine Reading Comprehension (MRC), as the name suggests, requires a machine to read a pas- sage and answer its relevant questions. Since the answer to each question is supposed to stem from the corresponding passage, a common MRC so- lution is to develop a neural-network-based MRC model that predicts an answer span (i.e. the an- swer start position and the answer end position) from the passage of each given passage-question pair. To facilitate the explorations and innovations in this area, many MRC datasets have been estab- lished, such as SQuAD (Rajpurkar et al., 2016), MS MARCO (Nguyen et al., 2016), and Trivi- aQA (Joshi et al., 2017). Consequently, many pi- oneering MRC models have been proposed, such as BiDAF (Seo et al., 2016), R-NET (Wang et al., 2017), and QANet (Yu et al., 2018). According to the leader board of SQuAD, the state-of-the-art MRC models have achieved the same performance as human beings. However, does this imply that they have possessed the same reading comprehen- sion ability as human beings? OF COURSE NOT. There is a huge gap between MRC models and human beings, which is mainly reflected in the hunger for data and the robust- ness to noise. On the one hand, developing MRC models requires a large amount of training exam- ples (i.e. the passage-question pairs labeled with answer spans), while human beings can achieve good performance on evaluation examples (i.e. the passage-question pairs to address) without training examples. On the other hand, Jia and Liang (2017) revealed that intentionally injected noise (e.g. mis- leading sentences) in evaluation examples causes the performance of MRC models to drop signif- icantly, while human beings are …

Content truncated to 500 words