Semi-supervised Domain Adaptation for Dependency Parsing

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2386–2395 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 2386 Semi-supervised Domain Adaptation for Dependency Parsing Zhenghua Li1, Xue Peng1, Min Zhang1∗, Rui Wang2, Luo Si2 1Institute of Artificial Intelligence, School of Computer Science and Technology, Soochow University 2Alibaba Group, China {zhli13, minzhang}@suda.edu.cn, 20175227031@stu.suda.edu.cn {masi.wr, luo.si}@alibaba-inc.com Abstract During the past decades, due to the lack of sufficient labeled data, most studies on cross- domain parsing focus on unsupervised domain adaptation, assuming there is no target- domain training data. However, unsupervised approaches make limited progress so far due to the intrinsic difficulty of both domain adaptation and parsing. This paper tackles the semi-supervised domain adaptation problem for Chinese dependency parsing, based on two newly-annotated large-scale domain-specific datasets.1 We propose a simple domain embedding approach to merge the source- and target-domain training data, which is shown to be more effective than both direct corpus concatenation and multi-task learning. In order to utilize unlabeled target-domain data, we employ the recent contextualized word representations and show that a simple fine-tuning procedure can further boost cross-domain parsing accuracy by large margins. 1 Introduction As a fundamental task in NLP, dependency parsing has attracted a lot of research interest during the past decades due to its multi-lingual appli- cability in capturing both syntactic and seman- tic information (K¨ubler et al., 2009; McDonald et al., 2013). Given an input sentence S w0w1 . . . wn, dependency parsing constructs a tree= d = {(h, m, l), 0 ≤ h ≤ n, 1 ≤ m ≤ n, l ∈ L}, as depicted in Figure 1, where (h, m, l) is a dependency from the head wh to the modifier wm ∗Corresponding author 1The two domain-specific datasets, plus another one for product comment texts, are also used in the NLPCC-2019 shared task (http://hlt.suda.edu.cn/index. php/Nlpcc-2019-shared-task) on cross-domain Chinese dependency parsing. Please note that the settings for the source-domain training data are different between this work and NLPCC-2019 shared task. $ ł Ł this with white shirt very pretty subj att adv adv obj root Figure 1: An example from the product blogs domain. The English translation is “This looks very pretty with a white shirt.” with the relation label l, and w0 is a pseudo root node. Recently, dependency parsing has achieved tremendous progress thanks to the strong capability of deep neural networks in capturing long-distance contexts (Chen and Manning, 2014; Dyer et al., 2015; Zhou et al., 2015; Andor et al., 2016; Kiperwasser and Goldberg, 2016; Dozat and Manning, 2017; Ma et al., 2018). Furthermore, contextualized word representations learned from large-scale unlabeled texts under language model training loss (Peters et al., 2018; Radford et al., 2018; Devlin et al., 2018), are proven to be extensively helpful for many NLP tasks including dependency parsing (Che et al., 2018; Clark et al., 2018; Kitaev and Klein, 2018). However, parsing performance drops dramati- cally when processing texts that are different from the training data, …

Content truncated to 500 words