Rethinking Label Smoothing on Multi-hop Question Answering

Rethinking Label Smoothing on Multi-hop Question Answering Zhangyue Yin♢∗ Yuxin Wang♢∗ Xiannian Hu♢ Yiguang Wu♢ Hang Yan♢ Xinyu Zhang♠ Zhao Cao♠ Xuanjing Huang♢ Xipeng Qiu♢† ♢School of Computer Science, Fudan University ♠Huawei Poisson Lab {yinzy21,wangyuxin21,xnhu21}@m.fudan.edu.cn {ygwu20,hyan19,xjhuang,xpqiu}@fudan.edu.cn {zhangxinyu35,caozhao1}@huawei.com Abstract Multi-Hop Question Answering (MHQA) is a significant area in question answering, requiring multiple reasoning components, including document retrieval, supporting sentence prediction, and answer span extraction. In this work, we present the first application of label smoothing to the MHQA task, aiming to enhance generalization capabilities in MHQA systems while miti- gating overfitting of answer spans and reasoning paths in the training set. We introduce a novel label smoothing technique, F1 Smoothing, which incorporates uncertainty into the learning pro- cess and is specifically tailored for Machine Reading Comprehension (MRC) tasks. Moreover, we employ a Linear Decay Label Smoothing Algorithm (LDLA) in conjunction with curricu- lum learning to progressively reduce uncertainty throughout the training process. Experiment on the HotpotQA dataset confirms the effectiveness of our approach in improving generaliza- tion and achieving significant improvements, leading to new state-of-the-art performance on the HotpotQA leaderboard. 1 Introduction Multi-Hop Question Answering (MHQA) is a rapidly evolving research area within question answering that involves answering complex questions by gathering information from multiple sources. This requires a model capable of performing several reasoning steps and handling diverse information structures. In recent years, MHQA has attracted significant interest from researchers due to its potential for addressing real-world problems. The mainstream approach to MHQA typically incorporates several components, including a document retriever, a supporting sentence selector, and a reading comprehension module (Tu et al., 2020; Wu et al., 2021; Li et al., 2022). These components collaborate to accurately retrieve and integrate relevant information from multiple sources, ultimately providing a precise answer to the given question. Despite the remarkable performance of modern MHQA models in multi-hop reasoning, they continue to face challenges with answer span errors and multi-hop reasoning errors. A study by S2G (Wu et al., 2021) reveals that the primary error source is answer span errors, constituting 74.55%, followed by multi-hop reasoning errors. This issue arises from discrepancies in answer span annotations between the training and validation sets. As illustrated in Figure 1(a), the training set answer includes the quantifier “times”, while the validation set answer does not. Upon examining 200 samples, we found that around 13.7% of answer spans in the HotpotQA validation set deviate from those in the training set. Concerning multi-hop reasoning, we identified the presence of unannotated, viable multi-hop reason- ing paths in the training set. As depicted in Figure 1(b), the non-gold document contains the necessary information to answer the question, similar to gold doc1, yet is labeled as an irrelevant document. Dur- ing training, the model can only discard this reasoning path and adhere to the annotated reasoning path. Given that current MHQA approaches primarily use cross-entropy loss for training multiple components, ∗Equal contribution. † Corresponding author. ©2023 China National Conference on Computational Linguistics Published under Creative Commons Attribution 4.0 International License CCL 2023 …

Content truncated to 500 words