Reranking and Self-Training for Parser Adaptation

Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the ACL, pages 337–344, Sydney, July 2006. c⃝2006 Association for Computational Linguistics Reranking and Self-Training for Parser Adaptation David McClosky, Eugene Charniak, and Mark Johnson Brown Laboratory for Linguistic Information Processing (BLLIP) Brown University Providence, RI 02912 {dmcc|ec|mj}@cs.brown.edu Abstract Statistical parsers trained and tested on the Penn Wall Street Journal (WSJ) treebank have shown vast improvements over the last 10 years. Much of this improvement, however, is based upon an ever-increasing number of features to be trained on (typi- cally) the WSJ treebank data. This has led to concern that such parsers may be too finely tuned to this corpus at the expense of portability to other genres. Such wor- ries have merit. The standard “Charniak parser” checks in at a labeled precision- recall f-measure of 89.7% on the Penn WSJ test set, but only 82.9% on the test set from the Brown treebank corpus. This paper should allay these fears. In par- ticular, we show that the reranking parser described in Charniak and Johnson (2005) improves performance of the parser on Brown to 85.2%. Furthermore, use of the self-training techniques described in (Mc- Closky et al., 2006) raise this to 87.8% (an error reduction of 28%) again with- out any use of labeled Brown data. This is remarkable since training the parser and reranker on labeled Brown data achieves only 88.4%. 1 Introduction Modern statistical parsers require treebanks to train their parameters, but their performance de- clines when one parses genres more distant from the training data’s domain. Furthermore, the tree- banks required to train said parsers are expensive and difficult to produce. Naturally, one of the goals of statistical parsing is to produce a broad-coverage parser which is rel- atively insensitive to textual domain. But the lack of corpora has led to a situation where much of the current work on parsing is performed on a sin- gle domain using training data from that domain — the Wall Street Journal (WSJ) section of the Penn Treebank (Marcus et al., 1993). Given the aforementioned costs, it is unlikely that many sig- nificant treebanks will be created for new genres. Thus, parser adaptation attempts to leverage ex- isting labeled data from one domain and create a parser capable of parsing a different domain. Unfortunately, the state of the art in parser portability (i.e. using a parser trained on one do- main to parse a different domain) is not good. The “Charniak parser” has a labeled precision-recall f-measure of 89.7% on WSJ but a lowly 82.9% on the test set from the Brown corpus treebank. Furthermore, the treebanked Brown data is mostly general non-fiction and much closer to WSJ than, e.g., medical corpora would be. Thus, most work on parser adaptation resorts to using some labeled in-domain data to fortify the larger quantity of out- of-domain data. In this paper, we present some encouraging re- sults on parser adaptation without any in-domain data. (Though we also present …

Content truncated to 500 words