Cross-Lingual Machine Reading Comprehension

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 1586–1595, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 1586 Cross-Lingual Machine Reading Comprehension Yiming Cui†‡, Wanxiang Che†, Ting Liu†, Bing Qin†, Shijin Wang‡§, Guoping Hu‡ †Research Center for Social Computing and Information Retrieval (SCIR), Harbin Institute of Technology, Harbin, China ‡State Key Laboratory of Cognitive Intelligence, iFLYTEK Research, China §iFLYTEK AI Research (Hebei), Langfang, China †{ymcui,car,tliu,qinb}@ir.hit.edu.cn ‡§{ymcui,sjwang3,gphu}@iflytek.com Abstract Though the community has made great progress on Machine Reading Comprehension (MRC) task, most of the previous works are solving English-based MRC problems, and there are few efforts on other languages mainly due to the lack of large-scale training data. In this paper, we propose Cross-Lingual Machine Reading Comprehension (CLMRC) task for the languages other than English. Firstly, we present several back-translation approaches for CLMRC task, which is straightforward to adopt. However, to accurately align the answer into another language is difficult and could introduce additional noise. In this context, we propose a novel model called Dual BERT, which takes advantage of the large-scale train- ing data provided by rich-resource language (such as English) and learn the semantic re- lations between the passage and question in a bilingual context, and then utilize the learned knowledge to improve reading comprehension performance of low-resource language. We conduct experiments on two Chinese machine reading comprehension datasets CMRC 2018 and DRCD. The results show consistent and significant improvements over various state- of-the-art systems by a large margin, which demonstrate the potentials in CLMRC task. 1 1 Introduction Machine Reading Comprehension (MRC) has been a popular task to test the reading ability of the machine, which requires to read text material and answer the questions based on it. Starting from cloze-style reading comprehension, various neural network approaches have been proposed and mas- sive progresses have been made in creating large- scale datasets and neural models (Hermann et al., 2015; Hill et al., 2015; Kadlec et al., 2016; Cui 1Resources available: https://github.com/ ymcui/Cross-Lingual-MRC. et al., 2017; Rajpurkar et al., 2016; Dhingra et al., 2017). Though various types of contributions had been made, most works are dealing with English reading comprehension. Reading comprehension in other than English has not been well-addressed mainly due to the lack of large-scale training data. To enrich the training data, there are two tradi- tional approaches. Firstly, we can annotate data by human experts, which is ideal and high-quality, while it is time-consuming and rather expensive. One can also obtain large-scale automatically gen- erated data (Hermann et al., 2015; Hill et al., 2015; Liu et al., 2017), but the quality is far beyond the usable threshold. Another way is to exploit cross-lingual approaches to utilize the data in rich- resource language to implicitly learn the relations between <passage, question, answer>. In this paper, we propose the Cross-Lingual Machine Reading Comprehension (CLMRC) task that aims to help reading comprehension in low- resource languages. First, we present several back-translation …

Content truncated to 500 words