Domain Adaptation for Syntactic and Semantic Dependency Parsing Using Deep Belief Networks

Domain Adaptation for Syntactic and Semantic Dependency Parsing Using Deep Belief Networks Haitong Yang, Tao Zhuang and Chengqing Zong National Laboratory of Pattern Recognition Institute of Automation, Chinese Academy of Sciences, Beijing, 100190, China {htyang, tao.zhuang, cqzong}@nlpr.ia.ac.cn Abstract In current systems for syntactic and seman- tic dependency parsing, people usually de- fine a very high-dimensional feature space to achieve good performance. But these systems often suffer severe performance drops on out- of-domain test data due to the diversity of fea- tures of different domains. This paper fo- cuses on how to relieve this domain adapta- tion problem with the help of unlabeled tar- get domain data. We propose a deep learning method to adapt both syntactic and semantic parsers. With additional unlabeled target do- main data, our method can learn a latent fea- ture representation (LFR) that is beneficial to both domains. Experiments on English data in the CoNLL 2009 shared task show that our method largely reduced the performance drop on out-of-domain test data. Moreover, we get a Macro F1 score that is 2.32 points higher than the best system in the CoNLL 2009 shared task in out-of-domain tests. 1 Introduction Both syntactic and semantic dependency parsing are the standard tasks in the NLP community. The state- of-the-art model performs well if the test data comes from the domain of the training data. But if the test data comes from a different domain, the perfor- mance drops severely. The results of the shared tasks of CoNLL 2008 and 2009 (Surdeanu et al., 2008; Hajiˇc et al., 2009) also substantiates the argument. To relieve the domain adaptation, in this paper, we propose a deep learning method for both syntactic and semantic parsers. We focus on the situation that, besides source domain training data and target do- main test data, we also have some unlabeled target domain data. Many syntactic and semantic parsers are devel- oped using a supervised learning paradigm, where each data sample is represented as a vector of fea- tures, usually a high-dimensional feature. The per- formance degradation on target domain test data is mainly caused by the diversity of features of differ- ent domains, i.e., many features in target domain test data are never seen in source domain training data. Previous work have shown that using word clus- ters to replace the sparse lexicalized features (Koo et al., 2008; Turian et al., 2010), helps relieve the performance degradation on the target domain. But for syntactic and semantic parsing, people also use a lot of syntactic features, i.e., features extracted from syntactic trees. For example, the relation path be- tween a predicate and an argument is a syntactic fea- ture used in semantic dependency parsing (Johans- son and Nugues, 2008). Figure 1 shows an exam- ple of this relation path feature. Obviously, syntac- tic features like this are also very sparse and usu- ally specific to each domain. The method of clus- tering fails in generalizing these kinds of features. Our method, however, is very different from clus- …

Content truncated to 500 words