Dependency parsing with structure preserving embeddings

Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics, pages 1684–1697 April 19 - 23, 2021. ©2021 Association for Computational Linguistics 1684 Dependency parsing with structure preserving embeddings ´Akos K´ad´ar∗ † Lan Xiao∗ Mete Kemertas‡ Federico Fancellu Allan Jepson Afsaneh Fazly Samsung AI Centre Toronto (SAIC Toronto) {lan.xiao, federico.f, allan.jepson, a.fazly}@samsung.com Abstract Modern neural approaches to dependency pars- ing are trained to predict a tree structure by jointly learning a contextual representation for tokens in a sentence, as well as a head– dependent scoring function. Whereas this strategy results in high performance, it is dif- ficult to interpret these representations in re- lation to the geometry of the underlying tree structure. Our work seeks instead to learn in- terpretable representations by training a parser to explicitly preserve structural properties of a tree. We do so by casting dependency pars- ing as a tree embedding problem where we in- corporate geometric properties of dependency trees in the form of training losses within a graph-based parser. We provide a thorough evaluation of these geometric losses, showing that the majority of them yield strong tree dis- tance preservation as well as parsing perfor- mance on par with a competitive graph-based parser (Qi et al., 2018). Finally, we show where parsing errors lie in terms of tree rela- tionship in order to guide future work. 1 Introduction Dependency grammars are syntactic formalisms that represent the syntactic structure of a sentence as asymmetric binary grammatical relations among words (Tesni`ere, 1959; Hudson, 1984; Melcuk, 2003). An example dependency structure is given in Figure 1. Formally, a dependency structure is defined as a directed graph where words are ver- tices and relations are labelled directed edges (the arcs) between a child (the dependent) and its par- ent (the head). In practice, dependency structures considered for syntactic analysis are trees. Depen- dency trees have long been used to improve the ∗Equal contribution. com).† Now at Borealis AI (akos.kadar@borealisai. ‡ Now at University of Toronto (kemertas@cs. toronto.edu). performance of many NLP applications, includ- ing machine translation (Ding and Palmer, 2004; Menezes et al., 2010; Bastings et al., 2017), re- lation extraction (Kambhatla, 2004; Bunescu and Mooney, 2005; Miwa and Bansal, 2016; Zhang et al., 2018), and semantic role labeling (Hacioglu, 2004; Marcheggiani and Titov, 2017; He et al., 2018). In order to assign the correct dependency tree to a sentence, dependency parsers are trained to cor- rectly identify head–dependent relations between pairs of words. Modern neural approaches do so by jointly learning contextual feature representations for the tokens in a sentence, as well as a parsing decision function. This is the case for recent graph- based parsers (Zhang et al., 2017; Dozat et al., 2017; Mohammadshahi and Henderson, 2020, inter alia) where an encoder feature extractor is comple- mented by a score function predicting the likeli- hood of a word to be the head of another. However, whereas this joint learning strategy results in state- of-the-art performance, the representations learned by these parsers …

Content truncated to 500 words