Sequence Labeling Parsing by Learning across Representations

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5350–5357 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 5350 Sequence Labeling Parsing by Learning Across Representations Michalina Strzyz David Vilares Carlos G´omez-Rodr´ıguez Universidade da Coru˜na, CITIC FASTPARSE Lab, LyS Research Group, Departamento de Computaci´on Campus de Elvi˜na, s/n, 15071 A Coru˜na, Spain {michalina.strzyz,david.vilares,carlos.gomez}@udc.es Abstract We use parsing as sequence labeling as a com- mon framework to learn across constituency and dependency syntactic abstractions. To do so, we cast the problem as multitask learning (MTL). First, we show that adding a parsing paradigm as an auxiliary loss consistently im- proves the performance on the other paradigm. Secondly, we explore an MTL sequence label- ing model that parses both representations, at almost no cost in terms of performance and speed. The results across the board show that on average MTL models with auxiliary losses for constituency parsing outperform single- task ones by 1.05 F1 points, and for depen- dency parsing by 0.62 UAS points. 1 Introduction Constituency (Chomsky, 1956) and dependency grammars (Mel’cuk, 1988; K¨ubler et al., 2009) are the two main abstractions for representing the syntactic structure of a given sentence, and each of them has its own particularities (Kahane and Mazziotta, 2015). While in constituency parsing the structure of sentences is abstracted as a phrase- structure tree (see Figure 1a), in dependency pars- ing the tree encodes binary syntactic relations be- tween pairs of words (see Figure 1b). When it comes to developing natural language processing (NLP) parsers, these two tasks are usu- ally considered as disjoint tasks, and their im- provements therefore have been obtained sepa- rately (Charniak, 2000; Nivre, 2003; Kiperwasser and Goldberg, 2016; Dozat and Manning, 2017; Ma et al., 2018; Kitaev and Klein, 2018). Despite the potential benefits of learning across representations, there have been few attempts in the literature to do this. Klein and Manning (2003) considered a factored model that provides separate methods for phrase-structure and lexical depen- dency trees and combined them to obtain optimal parses. With a similar aim, Ren et al. (2013) first compute the n best constituency trees using a prob- abilistic context-free grammar, convert those into dependency trees using a dependency model, com- pute a probability score for each of them, and fi- nally rerank the most plausible trees based on both scores. However, these methods are complex and intended for statistical parsers. Instead, we pro- pose a extremely simple framework to learn across constituency and dependency representations. Contribution (i) We use sequence labeling for constituency (G´omez-Rodr´ıguez and Vilares, 2018) and dependency parsing (Strzyz et al., 2019) combined with multi-task learning (MTL) (Caru- ana, 1997) to learn across syntactic representa- tions. To do so, we take a parsing paradigm (con- stituency or dependency parsing) as an auxiliary task to help train a model for the other parsing representation, a simple technique that translates into consistent improvements across the board. (ii) We also show that a single MTL model fol- lowing …

Content truncated to 500 words