Unsupervised Cross-Lingual Adaptation of Dependency Parsers Using CRF Autoencoders

Findings of the Association for Computational Linguistics: EMNLP 2020, pages 2127–2133 November 16 - 20, 2020. c⃝2020 Association for Computational Linguistics 2127 Unsupervised Cross-Lingual Adaptation of Dependency Parsers Using CRF Autoencoders Zhao Li and Kewei Tu∗ School of Information Science and Technology, ShanghaiTech University Shanghai Engineering Research Center of Intelligent Vision and Imaging {lizhao,tukw}@shanghaitech.edu.cn Abstract We consider the task of cross-lingual adapta- tion of dependency parsers without annotated target corpora and parallel corpora. Previous work either directly applies a discriminative source parser to the target language, ignoring unannotated target corpora, or employs an un- supervised generative parser that can leverage unannotated target data but has weaker repre- sentational power than discriminative parsers. In this paper, we propose to utilize unsuper- vised discriminative parsers based on the CRF autoencoder framework for this task. We train a source parser and use it to initialize and reg- ularize a target parser that is trained on unan- notated target data. We conduct experiments that transfer an English parser to 20 target lan- guages. The results show that our method sig- nificantly outperforms previous methods.1 1 Introduction Supervised learning of dependency parsing is dif- ficult for low-resource languages because of the lack of large treebanks. On the other hand, cross- lingual adaptation of dependency parsers from rich- resource languages to low-resource languages has shown a lot of promise (Hwa et al., 2005; Zeman and Resnik, 2008; McDonald et al., 2011; Xiao and Guo, 2014; Tiedemann, 2015; Schlichtkrull and Søgaard, 2017; Ahmad et al., 2019), especially with the help of cross-lingual word representation (Wu and Dredze, 2019) or part-of-speech (POS) tags (Guo et al., 2015). In this paper, we consider the scenario in which there is only unannotated data for the target lan- guage that is not parallel to the source language treebank. A simple strategy is zero-shot transfer or direct transfer, which trains a parser on the source ∗Corresponding Author 1Code is available at https://github.com/livc/ cross-crfae. treebank and then directly applies it to the target language (Schuster et al., 2019; Wang et al., 2019). In order to leverage unannotated target data, He et al. (2019) propose to employ an unsupervised generative parser that can be trained on the target data while also regularized via soft parameter tying by a source parser. However, generative parsers are known to underperform discriminative parsers in rich-resource scenarios, mostly because of the un- realistic independence assumptions typically made by generative parsers. In fact, He et al. (2019) show that when they use multilingual BERT (Ken- ton and Toutanova, 2019) as the cross-lingual word representation, their method underperforms direct transfer of a strong discriminative parser. In this paper, we propose to instead use an un- supervised discriminative parser based on the CRF autoencoder framework (Ammar et al., 2014; Cai et al., 2017) for cross-lingual parser adaptation. We perform supervised training of the source parser with the source treebank and then use it to ini- tialize the target parser. The target parser is then trained on the unannotated target data in an …

Content truncated to 500 words