Learning with Limited Data for Multilingual Reading Comprehension

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 2840–2850, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 2840 Learning with Limited Data for Multilingual Reading Comprehension Kyungjae Lee1∗ Sunghyun Park1∗ Hojae Han1 Jinyoung Yeo3 Seung-won Hwang1† Juho Lee2 1Yonsei University, 2 NAVER Corp, 3 SK T-Brain Abstract This paper studies the problem of supporting question answering in a new language with limited training resources. As an extreme sce- nario, when no such resource exists, one can (1) transfer labels from another language, and (2) generate labels from unlabeled data, us- ing translator and automatic labeling function respectively. However, these approaches in- evitably introduce noises to the training data, due to translation or generation errors, which require a judicious use of data with vary- ing confidence. To address this challenge, we propose a weakly-supervised framework that quantifies such noises from automatically gen- erated labels, to deemphasize or fix noisy data in training. On reading comprehension task, we demonstrate the effectiveness of our model on low-resource languages with varying simi- larity to English, namely, Korean and French. 1 Introduction Reading comprehension question answering (RCQA) is one of many well-known NLP tasks, to answer questions based on text understanding. For RCQA, a well-known resource is SQuAD (Ra- jpurkar et al., 2016) with 100K QA data created by human, followed by NarrativeQA (Koˇcisk`y et al., 2018), SQuAD 2.0 (Rajpurkar et al., 2018), and CoQA (Reddy et al., 2018). However, as these datasets support only English, supporting other languages requires either annotation efforts in a comparable scale (Lim et al., 2018), or modeling efforts to overcome the limitation of training resources in terms of quantity or quality. Our work pursues the latter goal. To illustrate, consider an extreme scenario of bootstrapping a RCQA model for a new language with no labelled resource. We can overcome the ∗First two authors equally contributed to this work. †correspond to seungwonh@yonsei.ac.kr limitation in quantity by generating alternative low-quality resources. First, neural machine trans- lation (NMT) can convert existing English anno- tations into a target language (Faruqui and Ku- mar, 2015; Ture and Boschee, 2016). For exam- ple, (passage p, question q, answer a) in SQuAD can be translated into (p′, q′, a′) in a target lan- guage (Asai et al., 2018; Lee et al., 2018). Second, automatic labeling function (Du and Cardie, 2018; Serban et al., 2016) can be adopted to generate synthetic labels in a target language. For example, Du and Cardie (2018) leverages Question Gener- ator (QG) and Answer Extractor (AE) as labeling function for RCQA, to create training resources of a virtually infinite amount from unlabeled corpora. However, overcoming the quantity limitation using the above methods, leads to quality prob- lems. As a consequence, some of the following assumptions on quality may cease to hold, after errors introduced from generation. (1) Answer- ability: the semantic of the translated passage or generated question may shift, so that a question …

Content truncated to 500 words