Robust Machine Reading Comprehension by Learning Soft labels

Proceedings of the 28th International Conference on Computational Linguistics, pages 2754–2759 Barcelona, Spain (Online), December 8-13, 2020 2754 Robust Machine Reading Comprehension by Learning Soft labels Zhenyu Zhao † Harbin Institute of Technology / Harbin, China zhaozhenyu1996@outlook.com Shuangzhi Wu, Tencent / Beijing, China frostwu@tencent.com Muyun Yang ‡ Harbin Institute of Technology / Harbin, China yangmuyun@hit.edu.cn Kehai Chen NICT / Kyoto, Japan khchen@nict.go.jp Tiejun Zhao Harbin Institute of Technology / Harbin, China tjzhao@hit.edu.cn Abstract Neural models have achieved great success on the task of machine reading comprehension (MRC), which are typically trained on hard labels. We argue that hard labels limit the model capability on generalization due to the label sparseness problem. In this paper, we propose a robust training method for MRC models to address this problem. Our method consists of three strategies, 1) label smoothing, 2) word overlapping, 3) distribution prediction. All of them help to train models on soft labels. We validate our approach on the representative architecture - ALBERT. Experimental results show that our method can greatly boost the baseline with 1% improvement in average, and achieve state-of-the-art performance on NewsQA and QUOREF. 1 Introduction Extractive reading comprehension is a challenging task in the field of natural language processing. Its ob- jective is, as shown in Fig.1, to detect a fragment from a passage to answer a given question. Our model needs to understand the context, locate the correct answer with exact word boundaries. Since multiple questions can be derived for a passage with different fragments of the words as the corresponding an- swers, the task is deemed as a benchmark for the deep understanding of human language, becoming a hot research topic in recent years. Usually, there exist multiple answers (or answer instances) for a given question in the extractive read- ing comprehension scenario. These answers generally express the same meaning with different combi- nation of words, or same expression but appear repeatedly in different locations, e.g. the multiple correct answers shown in Fig.1. However, the application of the standard cross-entropy loss in training allows only one correct answer to be considered, with other candidate answers ignored. Paragraph: One of the first Norman mercenaries to serve as a Byzantine general was Herv´e in the 1050s. By then however, there were already Norman mercenaries serving as far away as Trebizond and Georgia. ... Question: When did Herv´e serve as a Byzantine general? Answer1: 1050s Answer2: in the 1050s Figure 1: An example of multiple answer in extractive reading comprehension † Work done while the first author was an intern at Tencent. ‡ Corresponding author. This work is licensed under a Creative Commons Attribution 4.0 International Licence. Licence details: http://creativecommons.org/licenses/by/4.0 2755 A variety of methods have been proposed to address this issue (Xiong et al., 2017; Hu et al., 2018; Su et al., 2020). Most of them are focused on the training algorithm of MRC model, using reinforcement learning or modifying loss function to generalize the word overlapping between a candidate answer and the correct answer. Using this …

Content truncated to 500 words