MaP: A Matrix-based Prediction Approach to Improve Span Extraction in Machine Reading Comprehension

Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing, pages 687–695 December 4 - 7, 2020. c⃝2020 Association for Computational Linguistics 687 MaP: A Matrix-based Prediction Approach to Improve Span Extraction in Machine Reading Comprehension Huaishao Luo1∗, Yu Shi2, Ming Gong3, Linjun Shou3, Tianrui Li1 1School of Information Science and Technology, Southwest Jiaotong University 2Microsoft Cognitive Services Research Group 3Microsoft STCA NLP Group huaishaoluo@gmail.com, trli@swjtu.edu.cn {yushi,migon,lisho}@microsoft.com Abstract Span extraction is an essential problem in ma- chine reading comprehension. Most of the ex- isting algorithms predict the start and end po- sitions of an answer span in the given corre- sponding context by generating two probabil- ity vectors. In this paper, we propose a novel approach that extends the probability vector to a probability matrix. Such a matrix can cover more start-end position pairs. Precisely, to each possible start index, the method al- ways generates an end probability vector. Be- sides, we propose a sampling-based training strategy to address the computational cost and memory issue in the matrix training phase. We evaluate our method on SQuAD 1.1 and three other question answering benchmarks. Lever- aging the most competitive models BERT and BiDAF as the backbone, our proposed ap- proach can get consistent improvements in all datasets, demonstrating the effectiveness of the proposed method. 1 Introduction Machine reading comprehension (MRC), which re- quires the machine to answer comprehension ques- tions based on the given passage of text, has been studied extensively in the past decades (Liu et al., 2019). Due to the increase of various large-scale datasets (e.g., SQuAD (Rajpurkar et al., 2016) and MS MARCO (Nguyen et al., 2016)), and the en- hancement of pre-trained models (e.g., ELMo (Pe- ters et al., 2018), BERT (Devlin et al., 2019), and XLNet (Yang et al., 2019)), remarkable advance- ments have been made recently in this area. Among various MRC tasks, span extraction is one of the essential tasks. Given the context and question, the span extraction task is to extract a span of the most plausible text from the corresponding context as a ∗This work was done during the first author’s internship at Microsoft Passage: ...,Begun as a one-page journal in September 1876, the Scholastic magazine is issued twice monthly and ... Question: When did the Scholastic Magazine of Notre dame begin publishing? Answer: September 1876 condi�onal condi�onal p(end|start) p(end|start) p(start|end) p(start|end) Vector-Based Matrix-Based Figure 1: An illustration of a machine reading compre- hension framework. Most of previous works are vector- based approaches shown as the left part. Our matrix- based conditional approach is shown in the right part. In our setting, every start (or end) position has an end (or start) probability vector, which leads that the output probabilities is a matrix (best seen in color). candidate answer. Although there exist unanswer- able cases beyond the span extraction, the span- based task is still fundamental and significant in the MRC field. Previous methods used to predict …

Content truncated to 500 words