Semi-supervised Training Data Generation for Multilingual Question Answering

Semi-supervised Training Data Generation for Multilingual Question Answering Kyungjae Lee, Kyoungho Yoon, Sunghyun Park, Seung-won Hwang Yonsei University, Department of Computer Science Yonsei-ro 50, Seoul, South Korea {lkj0509, e05087, pshy410, seungwonh}@yonsei.ac.kr Abstract Recently, various datasets for question answering (QA) research have been released, such as SQuAD, Marco, WikiQA, MCTest, and SearchQA. However, such existing training resources for these task mostly support only English. In contrast, we study semi-automated creation of the Korean Question Answering Dataset (K-QuAD), by using automatically translated SQuAD and a QA system bootstrapped on a small QA pair set. As a na¨ıve approach for other language, using only machine-translated SQuAD shows limited performance due to translation errors. We study why such approach fails and motivate needs to build seed resources to enable leveraging such resources. Specifically, we annotate seed QA pairs of small size (4K) for Korean language, and design how such seed can be combined with translated English resources. These approach, by combining two resources, leads to 71.50 F1 on Korean QA (comparable to 77.3 F1 on SQuAD). Keywords: QA dataset, Machine Comprehension, Multilingual Resource 1. Introduction Understanding text and answering questions about the text is an important yet challenging task for machines. To train machines for doing that, large-scale resources are neces- sary. For English, SQuAD (Rajpurkar et al., 2016) provides more than 100K pairs of questions and supporting passages: For example, for the question, such as “what causes precip- itation to fall?”, they locate the supporting “span”, or the relevant answer found in the passage, such as “precipitation ... falls under gravity”. As the advent of deep learning tech- niques turns algorithms data hungry, building resources of a significant size is a critical task, question answering (QA) systems, as demonstrated not only by SQuAD, but also by ImageNet for object recognition (Deng et al., ). However, existing datasets for these tasks are mostly built on English. For other languages, building resources of large scale is labor intensive. Meanwhile, some approaches transfer English resources to resource-poor language for NLP tasks. As improvement of neural-based Machine Translation (MT) (Bahdanau et al., 2014) and availabil- ity for 100+ languages, some research takes advantage of MT systems for multilingual tasks such as sentiment analysis (Balahur and Turchi, 2014), query-document rel- evance (Ture and Boschee, 2014), and named entity recog- nition (Dandapat and Way, 2016). Given this opportunity, our research questions are as follow: • RQ1: Can we build QA resource for other language, solely by automatically translating English resources? • RQ2: When annotating QA resource manually for other language, can we save efforts by leveraging ex- isting resources for English? • RQ3: How can we combine the two to complement each other? Regarding RQ1, we study Machine-Translated resource of a SQuAD, denoted as MT. Assuming perfect translation, MT should be sufficient to train a QA system of compara- ble performance to English QA trained on SQuAD. How- ever, there are two challenges: (1) The position of answer span changes or is lost in translation. (2) Translation qual- ity …

Content truncated to 500 words