Project-then-Transfer: Effective Two-stage Cross-lingual Transfer for Semantic Dependency Parsing

Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics, pages 2586–2594 April 19 - 23, 2021. ©2021 Association for Computational Linguistics 2586 Project-then-Transfer: Effective Two-stage Cross-lingual Transfer for Semantic Dependency Parsing Hiroaki Ozaki, Gaku Morio, Terufumi Morishita, Toshinori Miyoshi Research and Development Group, Hitachi, Ltd., Japan {hiroaki.ozaki.yu, gaku.morio.vn, terufumi.morishita.wp, toshinori.miyoshi.pd}@hitachi.com Abstract This paper describes the first report on cross- lingual transfer for semantic dependency pars- ing. We present the insight that there are two different kinds of cross-linguality, namely sur- face level and semantic level, and try to cap- ture both kinds of cross-linguality by combin- ing annotation projection and model transfer of pre-trained language models. Our exper- iments showed that the performance of our graph-based semantic dependency parser al- most achieved the approximated upper bound. 1 Introduction Cross-lingual dependency parsing attracted much attention for its powerful representational capa- bility in grammatical and semantic lexical rela- tions (Zhang and Barzilay, 2015; Guo et al., 2015; Ammar et al., 2016; Zeman et al., 2017, 2018; de Lhoneux et al., 2018; Schuster et al., 2019). Several remarkable contributions have been made in syntactic dependency parsing, especially on uni- versal dependencies (UD; Nivre et al. 2016). For example, Kondratyuk and Straka (2019) showed that a single multilingually fine-tuned neural model utilizing a pre-trained language model could suc- cessfully parse 75 languages in UD with compara- ble performances to state-of-the-art parsers. However, cross-lingual semantic dependency parsing (Oepen et al., 2014, 2015, 2016), which is totally different dependency structure from syn- tactic dependencies (shown in Figure 1), has not been explored as far as we know. A reason for this is the lack of parallel graphbanks that cover many languages with consistent annotation policies. One exception is Prague Semantic Dependencies (PSD; Mikulov´a 2009), which is a treebank of bi-lexical semantic graphs and contains over 30,000 pairs of parallel annotated sentences from the Wall Street Journal in English and Czech. A technique is impossible to apply to crops ��� ����� ����� ���� ���� ��� ��� ������� ������� ������� �������� ������������� �������������� Figure 1: Example dependency annotations. Above: Semantic dependency (PSD). Below: Syntactic depen- dency (UD). Semantic dependency focuses more on meaning relationship between words. Considering these circumstances, we propose to train semantic dependency parsers by capturing commonalities across languages as a remedy for the absence of massive multilingual graphbanks. Our work draws on the intuition that cross-linguality exists in both superficial level and semantic level. Accordingly, we leverage a two-stage fashion in- volving treebank-based transfer and model-based transfer. Treebank-based transfer, often called annotation projection, is a method of projecting source lan- guage annotations to a target language by using a mapping function such as word alignment. The annotation projection has been reported as a promis- ing approach under truly low-resource settings for UD parsing (Rosa and Mareˇcek, 2018). However, annotation projection often suffers from noise in word alignment (Damonte and Cohen, 2018). For the model-based transfer, several studies on trans- ferring contextualized word vectors have reported that it improves the …

Content truncated to 500 words