Synthetic Data Augmentation for Zero-Shot Cross-Lingual Question Answering

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 7016–7030 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 7016 Synthetic Data Augmentation for Zero-Shot Cross-Lingual Question Answering Arij Riabi‡∗ Thomas Scialom⋆⋄∗ Rachel Keraron⋆ Benoît Sagot‡ Djamé Seddah‡ Jacopo Staiano⋆ ‡ Inria, Paris, France ⋄ Sorbonne Université, CNRS, LIP6, F-75005 Paris, France ⋆ reciTAL, Paris, France {thomas,rachel,jacopo}@recital.ai {arij.riabi,benoit.sagot,djame.seddah}@inria.fr Abstract Coupled with the availability of large scale datasets, deep learning architectures have en- abled rapid progress on Question Answering tasks. However, most of those datasets are in English, and the performances of state-of-the- art multilingual models are significantly lower when evaluated on non-English data. Due to high data collection costs, it is not realistic to obtain annotated data for each language one desires to support. We propose a method to improve Cross- lingual Question Answering performance without requiring additional annotated data, leveraging Question Generation models to pro- duce synthetic samples in a cross-lingual fash- ion. We show that the proposed method allows to significantly outperform the base- lines trained on English data only, establish- ing thus a new state-of-the-art on four multi- lingual datasets: MLQA, XQuAD, SQuAD-it and PIAF (fr). 1 Introduction Question Answering is a fast-growing research field, aiming to improve the capabilities of ma- chines to read and understand documents. Signifi- cant progress has recently been enabled by the use of large pre-trained language models (Devlin et al., 2019; Raffel et al., 2020), which reach human-level performances on several publicly available bench- marks, such as SQuAD (Rajpurkar et al., 2016) and NewsQA (Trischler et al., 2017). Given that the majority of large scale Question Answering (QA) datasets are in English (Hermann et al., 2015; Rajpurkar et al., 2016; Choi et al., 2018), the development of QA systems targeting other languages is currently addressed via two cross-lingual QA datasets: XQuAD (Artetxe et al., 2020) and MLQA (Lewis et al., 2020a), covering ∗: equal contribution. The work of Arij Riabi was partly carried out while she was working at reciTAL. respectively 10 and 7 languages. Due to the cost of annotation, both are limited only to an evalua- tion set. They are comparable to the validation set of the original SQuAD (see more details in Sec- tion 3.3). In both datasets, each paragraph is paired with questions in various languages, allowing to evaluate models in a cross-lingual experimental scenario: the input context and the question can be in two different languages. This scenario has important practical applications, such as querying a set of documents in various languages. Performing this cross-lingual task is complex and remains challenging for current models, as- suming only English training data: transfer results are shown to rank behind training-language perfor- mance (Artetxe et al., 2020; Lewis et al., 2020a). In other words, multilingual models fine-tuned only on English data are found to perform significantly better on English than on other languages. Be- sides the almost simultaneous work of Shakeri et al. (2020), very few alternatives to such a simple zero- shot transfer method have …

Content truncated to 500 words