X-SRL: A Parallel Cross-Lingual Semantic Role Labeling Dataset

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 3904–3914, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 3904 X-SRL: A Parallel Cross-Lingual Semantic Role Labeling Dataset Angel Daza†p and Anette Frankp Leibniz ScienceCampus “Empirical Linguistics and Computational Language Modeling” pHeidelberg University / †Institut f¨ur Deutsche Sprache Mannheim Germany {daza,frank}@cl.uni-heidelberg.de Abstract Even though SRL is researched for many lan- guages, major improvements have mostly been obtained for English, for which more resour- ces are available. In fact, existing multilin- gual SRL datasets contain disparate annotation styles or come from different domains, ham- pering generalization in multilingual learning. In this work we propose a method to automa- tically construct an SRL corpus that is paral- lel in four languages: English, French, Ger- man, Spanish, with unified predicate and role annotations that are fully comparable across languages. We apply high-quality machine translation to the English CoNLL-09 dataset and use multilingual BERT to project its high- quality annotations to the target languages. We include human-validated test sets that we use to measure the projection quality, and show that projection is denser and more precise than a strong baseline. Finally, we train different SOTA models on our novel corpus for mono- and multilingual SRL, showing that the multi- lingual annotations improve performance espe- cially for the weaker languages. 1 Introduction Semantic Role Labeling (SRL) is the task of ex- tracting semantic predicate-argument structures from sentences. One of the most widely used la- beling schemes for this task is based on PropBank (Palmer et al., 2005). It comes in two variants: span-based labeling, where arguments are char- acterized as word-spans (Carreras and M`arquez, 2005; Pradhan et al., 2012), and head-based label- ing, which only labels the syntactic head (Hajiˇc et al., 2009). In this work we focus on head-based labeling, as it is applied in the multilingual CoNLL- 09 shared task dataset, comprising 7 languages. The performance of English SRL has consider- ably improved in recent years through continuous Figure 1: Method to create X-SRL. We automatically translate the English CoNLL-09 corpus, use a fast label projection method for train-dev and get human annota- tors to select the appropriate head words on the target sentences to obtain gold annotations for the test sets. refinements of Deep Neural Network (DNN) mod- els (Zhou and Xu, 2015; He et al., 2017; Marcheg- giani et al., 2017; Cai et al., 2018); however, al- though the CoNLL-09 SRL dataset already covers 7 languages, other languages have not received the same level of attention. This situation may be due to factors such as i) the lack of sufficient training data to successfully apply a language- agnostic DNN model; ii) the fact that creating new SRL datasets is resource-consuming; iii) current label projection methods suffering from low re- call; finally, iv) even in cases where annotated re- sources are available in other languages, often they were automatically converted from independent pre-existing annotation schemes or labeled with automatic methods, resulting in data quality and labeling schema divergences, hampering …

Content truncated to 500 words