Semi-Supervised Dependency Parsing with Arc-Factored Variational Autoencoding

Proceedings of the 28th International Conference on Computational Linguistics, pages 2485–2496 Barcelona, Spain (Online), December 8-13, 2020 2485 Semi-Supervised Dependency Parsing with Arc-Factored Variational Autoencoding Ge Wang, Kewei Tu∗ School of Information Science and Technology, ShanghaiTech University, Shanghai, China Shanghai Engineering Research Center of Intelligent Vision and Imaging Shanghai Institute of Microsystem and Information Technology, Chinese Academy of Sciences University of Chinese Academy of Sciences {wangge,tukw}@shanghaitech.edu.cn Abstract Mannual annotation for dependency parsing is both labourious and time costly, resulting in the difficulty to learn practical dependency parsers for many languages due to the lack of labelled training corpora. To compensate for the scarcity of labelled data, semi-supervised dependency parsing methods are developed to utilize unlabelled data in the training procedure of dependency parsers. In previous work, the autoencoder framework is a prevalent approach for the utiliza- tion of unlabelled data. In this framework, training sentences are reconstructed from a decoder conditioned on dependency trees predicted by an encoder. The tree structure requirement brings challenges for both the encoder and the decoder. Sophisticated techniques are employed to tackle these challenges at the expense of model complexity and approximations in encoding and decod- ing. In this paper, we propose a model based on the variational autoencoder framework. By relaxing the tree constraint in both the encoder and the decoder during training, we make the learning of our model fully arc-factored and thus circumvent the challenges brought by the tree constraint. We evaluate our model on datasets across several languages and the results demon- strate the advantage of our model over previous approaches in both parsing accuracy and speed. 1 Introduction Dependency parsing is the task of finding syntactic dependency relations between words in sentences (K¨ubler et al., 2009). For each sentence, the dependency arcs between words are constrained to form a tree structure. A main bottleneck for learning a practical dependency parser is the lack of adequate training corpora as labelling raw text with dependency trees is both labourious and time costly. Semi- supervised dependency parsing utilizes unlabelled data to compensate the scarcity of labelled data in training dependency parsers (Sagae and Tsujii, 2007; Chen et al., 2009; Suzuki et al., 2011; Li et al., 2014). In previous work, the autoencoder framework is a prevalent approach for the utilization of unlabelled data (Ammar et al., 2014; Kingma and Welling, 2014). When this framework is applied to dependency parsing, training sentences are reconstructed from a decoder conditioned on dependency trees predicted by an encoder. Concretely, the autoencoder approaches to dependency parsing are mainly divided into two categories: the Conditional Random Field (CRF) Autoencoder (Cai et al., 2017) and the Variational Autoencoder (VAE) (Corro and Titov, 2019; Li et al., 2019). The CRF autoencoder predicts the de- pendency structure with the encoder and tries to reconstruct the input sentence based on the predicted structure. The variational autoencoder assumes the decoder is a generative model which generates the observed sentence from a group of latent variables containing information of the dependency tree, while the encoder …

Content truncated to 500 words