Semi-supervised Autoencoding Projective Dependency Parsing

Proceedings of the 28th International Conference on Computational Linguistics, pages 3868–3885 Barcelona, Spain (Online), December 8-13, 2020 3868 Semi-supervised Autoencoding Projective Dependency Parsing Xiao Zhang∗ Search Science & AI Amazon.com zhxao@amazon.com Dan Goldwasser Department of Computer Science Purdue University dgoldwas@purdue.edu Abstract We describe two end-to-end autoencoding models for semi-supervised graph-based projective dependency parsing. The first model is a Locally Autoencoding Parser (LAP) encoding the in- put using continuous latent variables in a sequential manner; The second model is a Globally Autoencoding Parser (GAP) encoding the input into dependency trees as latent variables, with exact inference. Both models consist of two parts: an encoder enhanced by deep neural networks (DNN) that can utilize the contextual information to encode the input into latent variables, and a decoder which is a generative model able to reconstruct the input. Both LAP and GAP admit a unified structure with different loss functions for labeled and unlabeled data with shared param- eters. We conducted experiments on WSJ and UD dependency parsing data sets, showing that our models can exploit the unlabeled data to improve the performance given a limited amount of labeled data, and outperform a previously proposed semi-supervised model. 1 Introduction Dependency parsing captures bi-lexical relationships by constructing directional arcs between words, defining a head-modifier syntactic structure for sentences, as shown in Figure 1. Dependency trees are fundamental for many downstream tasks such as semantic parsing [Reddy et al., 2016; Marcheggiani and Titov, 2017], machine translation [Bastings et al., 2017; Ding and Palmer, 2007], information extraction [Culotta and Sorensen, 2004; Liu et al., 2015] and question answering [Cui et al., 2005]. As a result, efficient parsers [Kiperwasser and Goldberg, 2016; Dozat and Manning, 2017; Dozat et al., 2017; Ma et al., 2018] have been developed using various neural architectures. PRP$ NN RB VBZ VBG NN PUNC My dog also likes eating sausage . root poss nsubj advmod xcomp dobj pu Figure 1: A dependency tree: directional arcs represent head-modifier relation between words. Despite the success of supervised approaches, they require large amounts of labeled data, particularly when neural architectures are used. Syntactic annotation is notoriously difficult and requires specialized linguistic expertise, posing a serious challenge for low-resource languages. Semi-supervised parsing aims to alleviate this problem by combining a small amount of labeled data and a large amount of unlabeled data, to improve parsing performance over labeled data alone. Traditional semi-supervised parsers use unlabeled data to generate additional features, assisting the learning process [Koo et al., ∗ This work was majorly done while the first author was pursuing a Ph.D. degree at Purdue University. This work is licensed under a Creative Commons Attribution 4.0 International License. License details: http:// creativecommons.org/licenses/by/4.0/. 3869 2008], together with different variants of self-training [Søgaard, 2010]. However, these are usually pipe- lined and error-propagation may occur. In this paper, we propose two end-to-end semi-supervised parsers, illustrated in Figure 2, namely Locally Autoencoding Parser (LAP) and Globally Autoencoding Parser (GAP). In LAP, the autoencoder model uses unlabeled examples to learn continuous latent variables …

Content truncated to 500 words