Automatic Spanish Translation of SQuAD Dataset for Multi-lingual Question Answering

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 5515–5523 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 5515 Automatic Spanish Translation of the SQuAD Dataset for Multilingual Question Answering Casimiro Pio Carrino, Marta R. Costa-juss`a, Jos´e A. R. Fonollosa TALP Research Center Universitat Polit`ecnica de Catalunya, Barcelona {casimiro.pio.carrino, marta.ruiz, jose.fonollosa}@upc.edu Abstract Recently, multilingual question answering became a crucial research topic, and it is receiving increased interest in the NLP community. However, the unavailability of large-scale datasets makes it challenging to train multilingual QA systems with performance comparable to the English ones. In this work, we develop the Translate Align Retrieve (TAR) method to automatically translate the Stanford Question Answering Dataset (SQuAD) v1.1 to Spanish. We then used this dataset to train Spanish QA systems by fine-tuning a Multilingual-BERT model. Finally, we evaluated our QA models with the recently proposed MLQA and XQuAD benchmarks for cross-lingual Extractive QA. Experimental results show that our models outperform the previous Multilingual-BERT baselines achieving the new state-of-the-art values of 68.1 F1 on the Spanish MLQA corpus and 77.6 F1 on the Spanish XQuAD corpus. The resulting, synthetically generated SQuAD-es v1.1 corpora, with almost 100% of data contained in the original English version, to the best of our knowledge, is the first large-scale QA training resource for Spanish. Keywords: Question Answering, Multilinguality, Corpus Creation 1. Introduction Question answering is a crucial and challenging task for machine-reading comprehension and represents a classical probe to assesses the ability of a machine to understand natural language (Hermann et al., 2015). In the last years, the field of QA has made enormous progress, primarily by fine-tuning deep pre-trained architectures (Vaswani et al., 2017; Devlin et al., 2018; Liu et al., 2019b; Yang et al., 2019) on large-scale QA datasets. Unfortunately, large and high-quality annotated corpora are usually scarce for languages other than English, hindering advancement in Multilingual QA research. Several approaches based on cross-lingual learning and synthetic corpora generation have been proposed. Cross- lingual learning refers to zero, and few-shot techniques applied to transfer the knowledge of a QA model trained on many source examples to a given target language with fewer training data. (Artetxe et al., 2019; Lee and Lee, 2019; Liu et al., 2019a) On the other hand, synthetic corpora generation methods are machine-translation (MT) based designed to automatically generate language-specific QA datasets as training resources (Alberti et al., 2019; Lee et al., 2018; T¨ure and Boschee, 2016). Additionally, a multilingual QA system based on MT at test time has also been explored (Asai et al., 2018) In this paper, we follow the synthetic corpora gen- eration approach. In particular, we developed the Translate-Align-Retrieve (TAR) method, based on MT and unsupervised alignment algorithm to translate an English QA dataset to Spanish automatically. Indeed, we applied our method to the popular SQuAD v1.1 generating its first Spanish version. We then trained two Spanish QA systems by fine-tuning the pre-trained Multilingual-BERT model. Finally, we evaluated our models on two Spanish …

Content truncated to 500 words