Statistical Ranking in Tactical Generation

Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing (EMNLP 2006), pages 517–525, Sydney, July 2006. c⃝2006 Association for Computational Linguistics Statistical Ranking in Tactical Generation Erik Velldal University of Oslo (Norway) erik.velldal@ifi.uio.no Stephan Oepen University of Oslo (Norway) and CSLI Stanford (CA) oe@csli.stanford.edu Abstract In this paper we describe and evaluate several statistical models for the task of realization ranking, i.e. the problem of discriminating between competing surface realizations generated for a given input se- mantics. Three models (and several vari- ants) are trained and tested: an n-gram language model, a discriminative maxi- mum entropy model using structural in- formation (and incorporating the language model as a separate feature), and finally an SVM ranker trained on the same feature set. The resulting hybrid tactical generator is part of a larger, semantic transfer MT system. 1 Introduction This paper describes the application of several dif- ferent statistical models for the task of realiza- tion ranking in tactical generation, i.e. the problem of choosing among multiple paraphrases that are generated for a given meaning representation. The specific realization component we use is the open- source chart generator of the Linguistic Knowl- edge Builder (LKB; Carroll, Copestake, Flickinger, & Poznanski, 1999; Carroll & Oepen, 2005). Given a meaning representation in the form of Minimal Recursion Semantics (MRS; Copestake, Flickinger, Malouf, Riehemann, & Sag, 1995), the generator outputs English realizations in ac- cordance with the HPSG LinGO English Resource Grammar (ERG; Flickinger, 2002). As an example of generator output, a sub-set of alternate realizations that are produced for a single input MRS is shown in Figure 1. For the two data sets considered in this paper, the aver- age number of realizations produced by the gen- erator is 85.7 and 102.2 (the maximum numbers are 4176 and 3408, respectively). Thus, there is immediate demand for a principled way of choos- ing a single output among the generated candi- dates. For this task we train and test three differ- ent statistical models: an n-gram language model, a maximum entropy model (MaxEnt) and a (lin- ear) support vector machine (SVM). These are all models that have proved popular within the NLP community, but it is usually only the first of these three that has been applied to the task of ranking in sentence generation. The latter two models that we present here go beyond the sur- face information used by the n-gram model, and are trained on a symmetric treebank with features defined over the full HPSG analyses of compet- ing realizations. Furthermore, such discriminative models are suitable for ‘on-line’ use within our generator—adopting the technique of selective un- packing from a packed forest (Carroll & Oepen, 2005)—which means our hybrid realizer obviates the need for exhaustive enumeration of candidate outputs. The present results extend our earlier work (Velldal, Oepen, & Flickinger, 2004)—and the related work of Nakanishi, Miyao, & Tsu- jii (2005)—to an enlarged data set, more feature types, and additional learners. The rest of this paper is structured as follows. …

Content truncated to 500 words