Dependency Parsing with Partial Annotations: An Empirical Comparison

Proceedings of the The 8th International Joint Conference on Natural Language Processing, pages 49–58, Taipei, Taiwan, November 27 – December 1, 2017 c⃝2017 AFNLP Dependency Parsing with Partial Annotations: An Empirical Comparison Yue Zhang†, Zhenghua Li†∗, Jun Lang‡, Qingrong Xia†, Min Zhang† †Soochow University, Suzhou, China {yzhang1107,qrxia}@stu.suda.edu.cn,{zhli13,minzhang}@suda.edu.cn ‡Alibaba Group, Hangzhou, China langjun.lj@alibaba-inc.com Abstract This paper describes and compares two straightforward approaches for depen- dency parsing with partial annotations (PA). The first approach is based on a forest-based training objective for two CRF parsers, i.e., a biaffine neural network graph-based parser (Biaffine) and a tradi- tional log-linear graph-based parser (LL- GPar). The second approach is based on the idea of constrained decoding for three parsers, i.e., a traditional linear graph- based parser (LGPar), a globally nor- malized neural network transition-based parser (GN3Par) and a traditional linear transition-based parser (LTPar). For the test phase, constrained decoding is also used for completing partial trees. We con- duct experiments on Penn Treebank under three different settings for simulating PA, i.e., random, most uncertain, and divergent outputs from the five parsers. The results show that LLGPar is most effective in di- rectly learning from PA, and other parsers can achieve best performance when PAs are completed into full trees by LLGPar. 1 Introduction Traditional supervised approaches for structural classification assume full annotation (FA), mean- ing that the training instances have complete manually-labeled structures. In the case of depen- dency parsing, FA means a complete parse tree is provided for each training sentence. However, re- cent studies suggest that it is more economic and effective to construct labeled data with partial an- notation (PA). A lot of research effort has been at- tracted to obtain partially-labeled data for different ∗Correspondence author $0 I1 saw2 Sarah3 with4 a5 telescope6 Figure 1: An example partial tree, where only the heads of “saw” and “with” are given. tasks via active learning (Sassano and Kurohashi, 2010; Mirroshandel and Nasr, 2011; Li et al., 2012; Marcheggiani and Arti`eres, 2014; Flannery and Mori, 2015; Li et al., 2016), cross-lingual syn- tax projection (Spreyer and Kuhn, 2009; Ganchev et al., 2009; Jiang et al., 2010; Li et al., 2014), or mining natural annotation implicitly encoded in web pages (Jiang et al., 2013; Liu et al., 2014; Nivre et al., 2014; Yang and Vozila, 2014). Fig- ure 1) gives an example sentence partially an- notated with two dependencies. However, there still lacks systematic study on how to build de- pendency parsers with PA. Most previous works listed above rely on ad-hoc strategies designed for only basic dependency parsers. One excep- tion is that Li et al. (2014) convert partial trees into forests and train a traditional log-linear graph- based dependency parser (LLGPar) with PA based on a forest-based objective, showing promising results. Meanwhile, it is still unclear how PAs can be used by other main-stream dependency parsers, such as the traditional linear graph-based parser (LGPar) and transition-based parser (LT- Par), and the newly proposed biaffine neural net- work graph-based parser (Biaffine) (Dozat and Manning, …

Content truncated to 500 words