Porting Statistical Parsers with Data-Defined Kernels

Proceedings of the 10th Conference on Computational Natural Language Learning (CoNLL-X), pages 6–13, New York City, June 2006. c⃝2006 Association for Computational Linguistics Porting Statistical Parsers with Data-Defined Kernels Ivan Titov University of Geneva 24, rue G´en´eral Dufour CH-1211 Gen`eve 4, Switzerland ivan.titov@cui.unige.ch James Henderson University of Edinburgh 2 Buccleuch Place Edinburgh EH8 9LW, United Kingdom james.henderson@ed.ac.uk Abstract Previous results have shown disappointing performance when porting a parser trained on one domain to another domain where only a small amount of data is available. We propose the use of data-defined ker- nels as a way to exploit statistics from a source domain while still specializing a parser to a target domain. A probabilistic model trained on the source domain (and possibly also the target domain) is used to define a kernel, which is then used in a large margin classifier trained only on the target domain. With a SVM classifier and a neural network probabilistic model, this method achieves improved performance over the probabilistic model alone. 1 Introduction In recent years, significant progress has been made in the area of natural language parsing. This re- search has focused mostly on the development of statistical parsers trained on large annotated corpora, in particular the Penn Treebank WSJ corpus (Marcus et al., 1993). The best statistical parsers have shown good results on this benchmark, but these statistical parsers demonstrate far worse results when they are applied to data from a different domain (Roark and Bacchiani, 2003; Gildea, 2001; Ratnaparkhi, 1999). This is an important problem because we cannot ex- pect to have large annotated corpora available for most domains. While identifying this problem, pre- vious work has not proposed parsing methods which are specifically designed for porting parsers. Instead they propose methods for training a standard parser with a large amount of out-of-domain data and a small amount of in-domain data. In this paper, we propose using data-defined ker- nels and large margin methods to specifically ad- dress porting a parser to a new domain. Data-defined kernels are used to construct a new parser which ex- ploits information from a parser trained on a large out-of-domain corpus. Large margin methods are used to train this parser to optimize performance on a small in-domain corpus. Large margin methods have demonstrated sub- stantial success in applications to many machine learning problems, because they optimize a mea- sure which is directly related to the expected test- ing performance. They achieve especially good per- formance compared to other classifiers when only a small amount of training data is available. Most of the large margin methods need the definition of a kernel. Work on kernels for natural language parsing has been mostly focused on the definition of kernels over parse trees (e.g. (Collins and Duffy, 2002)), which are chosen on the basis of domain knowledge. In (Henderson and Titov, 2005) it was proposed to apply a class of kernels derived from probabilistic models to the natural language parsing problem. In (Henderson and Titov, 2005), the kernel is …

Content truncated to 500 words