Semi-Supervised Semantic Dependency Parsing Using CRF Autoencoders

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6795–6805 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 6795 Semi-Supervised Semantic Dependency Parsing Using CRF Autoencoders Zixia Jia⋄, Youmi Ma†, Jiong Cai⋄, Kewei Tu⋄∗ ⋄School of Information Science and Technology, ShanghaiTech University Shanghai Institute of Microsystem and Information Technology, Chinese Academy of Sciences University of Chinese Academy of Sciences Shanghai Engineering Research Center of Intelligent Vision and Imaging †Tokyo Institute of Technology {jiazx, caijiong, tukw}@shanghaitech.edu.cn youmi.ma@nlp.c.titech.ac.jp Abstract Semantic dependency parsing, which aims to find rich bi-lexical relationships, allows words to have multiple dependency heads, resulting in graph-structured representations. We pro- pose an approach to semi-supervised learning of semantic dependency parsers based on the CRF autoencoder framework. Our encoder is a discriminative neural semantic dependency parser that predicts the latent parse graph of the input sentence. Our decoder is a gener- ative neural model that reconstructs the input sentence conditioned on the latent parse graph. Our model is arc-factored and therefore pars- ing and learning are both tractable. Experi- ments show our model achieves significant and consistent improvement over the supervised baseline. 1 Introduction Semantic dependency parsing (SDP) is a task aim- ing at discovering sentence-internal linguistic in- formation. The focus of SDP is the identification of predicate-argument relationships for all content words inside a sentence (Oepen et al., 2014, 2015). Compared with syntactic dependencies, semantic dependencies are more general, allowing a word to be either unattached or the argument of multi- ple predicates. The set of semantic dependencies within a sentence form a directed acyclic graph (DAG), distinguishing SDP from syntactic depen- dency parsing tasks, where dependencies are usu- ally tree-structured. Extraction of such high-level structured semantic information potentially ben- efits downstream NLP tasks (Reddy et al., 2017; Schuster et al., 2017). Several supervised SDP models are proposed in the recent years by modifying syntactic depen- dency parsers. Their parsing mechanisms are either transition-based (Kanerva et al., 2015; Wang et al., ∗Corresponding author. 2018) or graph-based (Martins and Almeida, 2014; Peng et al., 2017; Dozat and Manning, 2018; Wang et al., 2019). One limitation of supervised SDP is that labeled SDP data resources are limited in scale and diver- sity. Due to the rich relationships in SDP, the anno- tation of semantic dependency graphs is expensive and difficult, calling for professional linguists to de- sign rules and highly skilled annotators to annotate sentences. This limitation becomes more severe with the rise of deep learning, because neural ap- proaches are more data-hungry and susceptible to over-fitting when lacking training data. To allevi- ate this limitation, we investigate semi-supervised SDP capable of learning from both labeled and unlabeled data. While a lot of work has been done on super- vised SDP, the research of unsupervised and semi- supervised SDP is still lacking. Since parsing re- sults of semantic dependencies are DAGs with- out the tree-shape restriction, most existing suc- cessful unsupervised (Klein and Manning, 2004; I. Spitkovsky et al., 2010; Jiang et al., 2016; Cai et …

Content truncated to 500 words