Semi-Supervised Feature Transformation for Dependency Parsing

Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1303–1313, Seattle, Washington, USA, 18-21 October 2013. c⃝2013 Association for Computational Linguistics Semi-supervised Feature Transformation for Dependency Parsing Wenliang Chen†, Min Zhang†∗, and Yue Zhang‡ †School of Computer Science and Technology, Soochow University, China ‡Singapore University of Technology and Design, Singapore {wlchen, mzhang}@suda.edu.cn yue zhang@sutd.edu.sg Abstract In current dependency parsing models, con- ventional features (i.e. base features) defined over surface words and part-of-speech tags in a relatively high-dimensional feature space may suffer from the data sparseness problem and thus exhibit less discriminative power on unseen data. In this paper, we propose a novel semi-supervised approach to address- ing the problem by transforming the base fea- tures into high-level features (i.e. meta fea- tures) with the help of a large amount of au- tomatically parsed data. The meta features are used together with base features in our final parser. Our studies indicate that our proposed approach is very effective in processing un- seen data and features. Experiments on Chi- nese and English data sets show that the fi- nal parser achieves the best-reported accuracy on the Chinese data and comparable accuracy with the best known parsers on the English data. 1 Introduction In recent years, supervised learning models have achieved lots of progress in the dependency pars- ing task, as can be found in the CoNLL shared tasks (Buchholz and Marsi, 2006; Nivre et al., 2007). The supervised models take annotated data as training data, utilize features defined over surface words, part-of-speech tags, and dependency trees, and learn the preference of features via adjusting feature weights. ∗Corresponding author In the supervised learning scenarios, many previ- ous studies explore rich feature representation that leads to significant improvements. McDonald and Pereira (2006) and Carreras (2007) define second- order features over two adjacent arcs in second- order graph-based models. Koo and Collins (2010) use third-order features in a third-order graph-based model. Bohnet (2010) considers information of more surrounding words for the graph-based mod- els, while Zhang and Nivre (2011) define a set of rich features including the word valency and the third-order context features for transition-based models. All these models utilize richer and more complex feature representations and achieve better performance than the earlier models that utilize the simpler features (McDonald et al., 2005; Yamada and Matsumoto, 2003; Nivre and Scholz, 2004). However, the richer feature representations result in a high-dimensional feature space. Features in such a space may suffer from the data sparseness problem and thus have less discriminative power on unseen data. If input sentences contain unknown features that are not included in training data, the parsers can usually give lower accuracy. Several methods have been proposed to alleviate this problem by using large amounts of unannotated data, ranging from self-training and co-training (Mc- Closky et al., 2006; Sagae and Tsujii, 2007) to more complex methods that collect statistical information from unannotated sentences and use them as addi- tional features (Koo et al., 2008; Chen et al., 2009). In …

Content truncated to 500 words