Improving Zero-Shot Cross-lingual Transfer for Multilingual Question Answering over Knowledge Graph

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5822–5834 June 6–11, 2021. ©2021 Association for Computational Linguistics 5822 Improving Zero-Shot Cross-lingual Transfer for Multilingual Question Answering over Knowledge Graph Yucheng Zhou1∗, Xiubo Geng2, Tao Shen3, Wenqiang Zhang1†, Daxin Jiang2† 1Fudan University, Shanghai, China 2Microsoft, Beijing, China 3Australian AI Institute, School of CS, FEIT, University of Technology Sydney {yczhou18,wqzhang}@fudan.edu.cn, tao.shen@student.uts.edu.au {xigeng,djiang}@microsoft.com Abstract Multilingual question answering over knowl- edge graph (KGQA) aims to derive answers from a knowledge graph (KG) for questions in multiple languages. To be widely applicable, we focus on its zero-shot transfer setting. That is, we can only access training data in a high- resource language, while need to answer mul- tilingual questions without any labeled data in target languages. A straightforward approach is resorting to pre-trained multilingual mod- els (e.g., mBERT) for cross-lingual transfer, but there is a still significant gap of KGQA performance between source and target lan- guages. In this paper, we exploit unsupervised bilingual lexicon induction (BLI) to map train- ing questions in source language into those in target language as augmented training data, which circumvents language inconsistency be- tween training and inference. Furthermore, we propose an adversarial learning strategy to al- leviate syntax-disorder of the augmented data, making the model incline to both language- and syntax-independence. Consequently, our model narrows the gap in zero-shot cross- lingual transfer. Experiments on two multi- lingual KGQA datasets with 11 zero-resource languages verify its effectiveness. 1 Introduction With the advance of large-scale human-curated knowledge graphs (KG), e.g., DBpedia (Auer et al., 2007) and Freebase (Bollacker et al., 2008), ques- tion answering over knowledge graph (KGQA) has become a crucial natural language processing (NLP) task to answer factoid questions. It has been integrated into real-world applications like search engines and personal assistants, so it attracts more attention from both academia and industry (Liang et al., 2017; Hu et al., 2018; Shen et al., 2019). Recently, a rising demand of KGQA systems is to answer the multilingual questions, motivating us ∗Work is done during internship at Microsoft. †Corresponding authors. to focus on multilingual KGQA. However, building a large-scale KG, as well as annotating QA data, is costly for each new language, not to mention many minority languages with a few native annota- tors. Therefore, we adopt a zero-shot cross-lingual transfer setting – a KGQA model is developed to perform inference on multilingual questions with the only access to training data and associated KG in a high-resource language (e.g., English). Providing the success of pre-trained monolin- gual encoders (Peters et al., 2018; Liu et al., 2019), some works (e.g., mBERT (Devlin et al., 2019) and XLM-R (Conneau et al., 2020)) pre-train a Trans- former encoder (Vaswani et al., 2017) on large- scale non-parallel multilingual corpora in a self- supervised manner. Then given an NLP task, a general paradigm for zero-shot cross-lingual trans- fer is to fine-tune a pre-trained multilingual encoder on the data in a data-rich (source) language. And …

Content truncated to 500 words