Improving Machine Reading Comprehension with General Reading Strategies

Proceedings of NAACL-HLT 2019, pages 2633–2643 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 2633 Improving Machine Reading Comprehension with General Reading Strategies Kai Sun1∗ Dian Yu2 Dong Yu2 Claire Cardie1 1Cornell University, Ithaca, NY, USA 2Tencent AI Lab, Bellevue, WA, USA ks985@cornell.edu, {yudian, dyu}@tencent.com cardie@cs.cornell.edu Abstract Reading strategies have been shown to im- prove comprehension levels, especially for readers lacking adequate prior knowledge. Just as the process of knowledge accumu- lation is time-consuming for human readers, it is resource-demanding to impart rich gen- eral domain knowledge into a deep language model via pre-training. Inspired by reading strategies identified in cognitive science, and given limited computational resources – just a pre-trained model and a fixed number of training instances – we propose three gen- eral strategies aimed to improve non-extractive machine reading comprehension (MRC): (i) BACK AND FORTH READING that consid- ers both the original and reverse order of an input sequence, (ii) HIGHLIGHTING, which adds a trainable embedding to the text embed- ding of tokens that are relevant to the ques- tion and candidate answers, and (iii) SELF- ASSESSMENT that generates practice ques- tions and candidate answers directly from the text in an unsupervised manner. By fine-tuning a pre-trained language model (Radford et al., 2018) with our pro- posed strategies on the largest general domain multiple-choice MRC dataset RACE, we obtain a 5.8% absolute increase in accuracy over the previous best result achieved by the same pre-trained model fine-tuned on RACE without the use of strategies. We further fine-tune the resulting model on a target MRC task, leading to an absolute improvement of 6.2% in average accuracy over previous state- of-the-art approaches on six representative non-extractive MRC datasets from different domains (i.e., ARC, OpenBookQA, MCTest, SemEval-2018 Task 11, ROCStories, and MultiRC). These results demonstrate the effectiveness of our proposed strategies and the versatility and general applicability of ∗This work was done when K. S. was an intern at the Tencent AI Lab, Bellevue, WA. our fine-tuned models that incorporate these strategies. Core code is available at https: //github.com/nlpdata/strategy/. 1 Introduction Recent years have seen a growing interest in ma- chine reading comprehension (MRC) (Rajpurkar et al., 2016; Choi et al., 2018; Koˇcisk`y et al., 2018; Reddy et al., 2018). In this paper, we mainly fo- cus on non-extractive MRC (Khashabi et al., 2018; Ostermann et al., 2018; Clark et al., 2018), in which a significant percentage of candidate an- swers are not restricted to text spans from the ref- erence document or corpus. In comparison to ex- tractive MRC tasks (Section 2.1), non-extractive MRC (Section 2.2) requires diverse reading skills and, as a result, the performance of machine read- ers on these tasks more accurately indicates the comprehension ability of machine readers in re- alistic settings such as exams (Lai et al., 2017). Recently, significant progress has been achieved on many natural language processing tasks includ- ing MRC by fine-tuning a pre-trained general- purpose language model (Radford et al., 2018; De- vlin et al., …

Content truncated to 500 words