Multitask Learning for Cross-Lingual Transfer of Broad-coverage Semantic Dependencies

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 8268–8274, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 8268 Multitask Learning for Cross-Lingual Transfer of Broad-coverage Semantic Dependencies Maryam Aminian1, Mohammad Sadegh Rasooli2, Mona Diab1,3 1Department of Computer Science, The George Washington University, Washington D.C. 2Department of Computer and Information Science, University of Pennsylvania, Philadelphia, PA 3Facebook AI, Seattle, WA, USA 1{aminian,mtdiab}@gwu.edu, 2rasooli@seas.upenn.edu Abstract We describe a method for developing broad- coverage semantic dependency parsers for lan- guages for which no semantically annotated re- source is available. We leverage a multitask learning framework coupled with annotation projection. We use syntactic parsing as the auxiliary task in our multitask setup. Our an- notation projection experiments from English to Czech show that our multitask setup yields 3.1% (4.2%) improvement in labeled F1-score on in-domain (out-of-domain) test set com- pared to a single-task baseline. 1 Introduction Broad-coverage semantic dependency parsing (SDP)1 was first introduced in the SemEval shared task (Oepen et al., 2014) and aims to provide se- mantic analysis of sentences by capturing semantic relations between all content-bearing words in a sentence. The rich graph structure introduced by SDP allows the model to cover a wide range of se- mantic phenomena such as negation, comparatives, possessives and various types of modifications that have not been previously analyzed in other models such as semantic role labeling (Baker et al., 1998). Despite all advantages provided by SDP, re- sources with annotated semantic dependencies are limited to the three languages released in the Se- mEval shared tasks (Oepen et al., 2014, 2015; Che et al., 2016) namely English, Czech and Chinese. This data scarcity motivates us to use well-known and traditionally used transfer methods such as an- notation projection for building SDP models for languages without semantically annotated data. In annotation projection, we assume that we have ac- cess to sentence-aligned corpora that can be used for transferring semantic annotations from a rich- resource source language to the target language 1We use broad-coverage semantic dependencies and se- mantic dependencies interchangeably throughout this paper. The sitting was closed at midnight zasedání bylo ukonˇceno o p˚ulnoci TOP PAT-arg TWHEN TOP PAT-arg TWHEN Figure 1: Projecting SDP annotations from an English to a Czech sentence. Semantic dependencies of the English sentence (top) are projected using alignments (dashed lines in the middle) to obtain projected seman- tic dependencies (bottom) for the target sentence. through word alignment links. Figure 1 shows an example of annotation projection for semantic de- pendencies. Motivated by the large amount of similarities between syntactic and semantic dependencies, we further propose a simple but effective multitask learning framework to leverage supervised syntac- tic parse information and improve the representa- tion learning capability in the intermediate layers of our semantic parser. Our multitask learning approach, despite its simplicity, yields significant improvements in the performance of the vanilla semantic dependency parser built using annotation projection. We conducted annotation projection experiments from English to Czech. Our experi- ments show that our multitask setup yields 3.1% …

Content truncated to 500 words