Target Language-Aware Constrained Inference for Cross-lingual Dependency Parsing

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 1117–1128, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 1117 Target Language-Aware Constrained Inference for Cross-lingual Dependency Parsing Tao Meng University of California, Los Angeles tmeng@cs.ucla.edu Nanyun Peng University of Southern California npeng@isi.edu Kai-Wei Chang University of California, Los Angeles kw@kwchang.net Abstract Prior work on cross-lingual dependency pars- ing often focuses on capturing the common- alities between source and target languages and overlooks the potential of leveraging lin- guistic properties of the languages to facil- itate the transfer. In this paper, we show that weak supervisions of linguistic knowl- edge for the target languages can improve a cross-lingual graph-based dependency parser substantially. Specifically, we explore several types of corpus linguistic statistics and com- pile them into corpus-wise constraints to guide the inference process during the test time. We adapt two techniques, Lagrangian relaxation and posterior regularization, to conduct infer- ence with corpus-statistics constraints. Ex- periments show that the Lagrangian relaxation and posterior regularization inference improve the performances on 15 and 17 out of 19 target languages, respectively. The improvements are especially significant for target languages that have different word order features from the source language. 1 Introduction Natural language processing (NLP) techniques have achieved remarkable performance in a vari- ety of tasks when sufficient training data is avail- able. However, obtaining high-quality annota- tions for low-resource language tasks is challeng- ing, and this poses great challenges to process low-resource languages. To bridge the gap, cross- lingual transfer has been proposed to transfer mod- els trained on high-resource languages (e.g., En- glish) to low-resource languages (e.g., Tamil) to combat the resource scarcity problem. Recent studies have demonstrated successes of transfer- ring models across languages without retraining for NLP tasks, such as named entity recognition (Xie et al., 2018), dependency parsing (Tiede- mann, 2015; Agi´c et al., 2014), and question answering (Joty et al., 2017), using a shared multi-lingual word embedding space (Smith et al., 2017) or delexicalization approaches (Zeman and Resnik, 2008; McDonald et al., 2013). One key challenge for cross-lingual transfer is the differences among languages; for example, languages may have different word orders. When transferring a model learned from a source lan- guage to target languages, the performance may drop significantly due to the differences. To tackle this problem, various approaches have been pro- posed to better capture the commonalities between the source and the target languages (McDonald et al., 2011; Guo et al., 2016; T¨ackstr¨om et al., 2013; Agi´c, 2017; Ahmad et al., 2019); however, they overlook the potential to leverage linguistic knowledge about the target language to account for the differences between the source and the tar- get languages to facilitate the transfer. In this paper, we propose a complementary ap- proach that studies how to leverage the linguis- tic knowledge about the target languages to help the transfer. Specifically, we use corpus linguistic statistics of the target languages as weak …

Content truncated to 500 words