Multi-View Cross-Lingual Structured Prediction with Minimum Supervision

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 2661–2674 August 1–6, 2021. ©2021 Association for Computational Linguistics 2661 Multi-View Cross-Lingual Structured Prediction with Minimum Supervision Zechuan Hu⋄‡, Yong Jiang†∗, Nguyen Bach†, Tao Wang†, Zhongqiang Huang†, Fei Huang†, Kewei Tu⋄∗ ⋄School of Information Science and Technology, ShanghaiTech University ⋄Shanghai Engineering Research Center of Intelligent Vision and Imaging ⋄Shanghai Institute of Microsystem and Information Technology, Chinese Academy of Sciences ⋄University of Chinese Academy of Sciences †DAMO Academy, Alibaba Group {huzch,tukw}@shanghaitech.edu.cn, yongjiang.jy@alibaba-inc.com Abstract In structured prediction problems, cross- lingual transfer learning is an efficient way to train quality models for low-resource lan- guages, and further improvement can be ob- tained by learning from multiple source lan- guages. However, not all source models are created equal and some may hurt performance on the target language. Previous work has ex- plored the similarity between source and tar- get sentences as an approximate measure of strength for different source models. In this paper, we propose a multi-view framework, by leveraging a small number of labeled tar- get sentences, to effectively combine multi- ple source models into an aggregated source view at different granularity levels (language, sentence, or sub-structure), and transfer it to a target view based on a task-specific model. By encouraging the two views to interact with each other, our framework can dynamically ad- just the confidence level of each source model and improve the performance of both views during training. Experiments for three struc- tured prediction tasks on sixteen data sets show that our framework achieves significant im- provement over all existing approaches, in- cluding these with access to additional source language data. 1 Introduction Structured prediction is the task of mapping input sentences to structured outputs. It is a fundamental task in natural language processing and has many applications, i.e., sequence labeling (DeRose, 1988; Lample et al., 2016; Ma and Hovy, 2016; Hu et al., 2020b), dependency parsing (Chen and Manning, 2014; Dozat and Manning, 2016; Ahmad et al., 2019) and semantic role labeling (van der Plas et al., 2011; Strubell et al., 2018; Cai and Lapata, 2020). ∗Corresponding authors. ‡Work was done when Zechuan Hu was interning at Alibaba DAMO Academy. To achieve strong performance, structured predic- tion models mostly require manually labeled data that are costly to obtain in general. Cross-lingual transfer learning (Yarowsky and Ngai, 2001; Wang and Manning, 2014; Guo et al., 2018; Lin et al., 2019; Hu et al., 2021) recently attracted attention for tackling that problem, by transferring the knowledge from high-resource lan- guages to low-resource ones. Existing works can be categorized into two types: single-source transfer and multi-source transfer. The former is limited to transferring knowledge from one source language and generally results in inferior performance than the latter (McDonald et al., 2011; Rahimi et al., 2019), especially when the target language is simi- lar to multiple source language over various char- acteristics, i.e., domain, word order, capitalization, and script style. …

Content truncated to 500 words