On the Benefit of Syntactic Supervision for Cross-lingual Transfer in Semantic Role Labeling

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6229–6246 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 6229 On the Benefit of Syntactic Supervision for Cross-lingual Transfer in Semantic Role Labeling Zhisong Zhang, Emma Strubell, Eduard Hovy Language Technologies Institute, Carnegie Mellon University zhisongz@cs.cmu.edu, strubell@cmu.edu, hovy@cmu.edu Abstract Although recent developments in neural archi- tectures and pre-trained representations have greatly increased state-of-the-art model perfor- mance on fully-supervised semantic role la- beling (SRL), the task remains challenging for languages where supervised SRL training data are not abundant. Cross-lingual learn- ing can improve performance in this setting by transferring knowledge from high-resource languages to low-resource ones. Moreover, we hypothesize that annotations of syntactic dependencies can be leveraged to further fa- cilitate cross-lingual transfer. In this work, we perform an empirical exploration of the helpfulness of syntactic supervision for cross- lingual SRL within a simple multitask learn- ing scheme. With comprehensive evaluations across ten languages (in addition to English) and three SRL benchmark datasets, including both dependency- and span-based SRL, we show the effectiveness of syntactic supervision in low-resource scenarios. 1 Introduction The task of semantic role labeling (SRL) annotates predicate-argument structures in text and is thus a desirable output of natural language processing (NLP) pipelines designed to extract information from text (Gildea and Jurafsky, 2002; Palmer et al., 2010). Recent developments in neural architectures (Vaswani et al., 2017) and pre-trained contextual- ized representations (Devlin et al., 2019; Liu et al., 2019) have greatly improved the performance of SRL systems (Zhou and Xu, 2015; He et al., 2017; Tan et al., 2018; Shi and Lin, 2019). However, most previous work focuses on high-resource En- glish SRL scenarios, and it remains a challenge to extend these approaches, which require plenti- ful supervised examples, to other languages where training resources may be limited. A popular approach addressing this challenge is cross-lingual learning: leveraging the shared struc- tures across human languages to transfer knowl- edge from high-resource languages to low-resource ones. Model transfer, where an SRL model is di- rectly transferred across languages using shared representations (Kozhevnikov and Titov, 2013, 2014; Fei et al., 2020b), is a particularly promising approach thanks to recent developments in multi- lingual contextualized representations (Lample and Conneau, 2019; Conneau et al., 2020), which have proven effective for cross-lingual transfer (Wu and Dredze, 2019; Pires et al., 2019). Another common strategy for improving SRL model performance in both high- and low-resource scenarios is incorporating syntactic information. Syntactic analysis was until recently considered a prerequisite for most SRL systems (Gildea and Palmer, 2002; Punyakanok et al., 2008) and has been shown to benefit recent neural models as well (Marcheggiani and Titov, 2017; He et al., 2018; Swayamdipta et al., 2018; Strubell et al., 2018). Despite much work exploring cross-lingual learn- ing and incorporating syntactic information into SRL systems, most such previous work explores these two avenues separately, though there are nu- merous reasons that carefully incorporating syntax into a cross-lingual system for SRL could provide further benefits: First, whereas …

Content truncated to 500 words