Training dependency parsers by jointly optimizing multiple objectives

Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, pages 1489–1499, Edinburgh, Scotland, UK, July 27–31, 2011. c⃝2011 Association for Computational Linguistics Training dependency parsers by jointly optimizing multiple objectives Keith Hall Ryan McDonald Jason Katz-Brown Michael Ringgaard Google Research {kbhall|ryanmcd|jasonkb|ringgaard}@google.com Abstract We present an online learning algorithm for training parsers which allows for the inclusion of multiple objective functions. The primary example is the extension of a standard su- pervised parsing objective function with addi- tional loss-functions, either based on intrinsic parsing quality or task-specific extrinsic mea- sures of quality. Our empirical results show how this approach performs for two depen- dency parsing algorithms (graph-based and transition-based parsing) and how it achieves increased performance on multiple target tasks including reordering for machine translation and parser adaptation. 1 Introduction The accuracy and speed of state-of-the-art depen- dency parsers has motivated a resumed interest in utilizing the output of parsing as an input to many downstream natural language processing tasks. This includes work on question answering (Wang et al., 2007), sentiment analysis (Nakagawa et al., 2010), MT reordering (Xu et al., 2009), and many other tasks. In most cases, the accuracy of parsers de- grades when run on out-of-domain data (Gildea, 2001; McClosky et al., 2006; Blitzer et al., 2006; Petrov et al., 2010). But these accuracies are mea- sured with respect to gold-standard out-of-domain parse trees. There are few tasks that actually depend on the complete parse tree. Furthermore, when eval- uated on a downstream task, often the optimal parse output has a model score lower than the best parse as predicted by the parsing model. While this means that we are not properly modeling the downstream task in the parsers, it also means that there is some information from small task or domain-specific data sets which could help direct our search for optimal parameters during parser training. The goal being not necessarily to obtain better parse performance, but to exploit the structure induced from human la- beled treebank data while targeting specific extrinsic metrics of quality, which can include task specific metrics or external weak constraints on the parse structure. One obvious approach to this problem is to em- ploy parser reranking (Collins, 2000). In such a setting, an auxiliary reranker is added in a pipeline following the parser. The standard setting involves training the base parser and applying it to a devel- opment set (this is often done in a cross-validated jack-knife training framework). The reranker can then be trained to optimize for the downstream or extrinsic objective. While this will bias the reranker towards the target task, it is limited by the oracle performance of the original base parser. In this paper, we propose a training algorithm for statistical dependency parsers (K¨ubler et al., 2009) in which a single model is jointly optimized for a regular supervised training objective over the tree- bank data as well as a task-specific objective – or more generally an extrinsic objective – on an ad- ditional data set. …

Content truncated to 500 words