Linguistically Informed Statistical Models of Constituent Structure for Ordering in Sentence Realization

Linguistically Informed Statistical Models of Constituent Structure for Ordering in Sentence Realization Eric RINGGER1, Michael GAMON1, Robert C. MOORE1, David ROJAS2, Martine SMETS1, Simon CORSTON-OLIVER1 1Microsoft Research One Microsoft Way Redmond, Washington 98052, USA {ringger, mgamon, bobmoore, msmets, simonco}@microsoft.com 2Butler Hill Group, LLC & Indiana University Linguistics Dept. 1021 East 3rd Street, MM 322 Bloomington, Indiana 47405, USA drojas@indiana.edu Abstract We present several statistical models of syntactic constituent order for sentence realization. We compare several models, including simple joint models inspired by existing statistical parsing models, and several novel conditional models. The conditional models leverage a large set of linguistic features without manual feature selection. We apply and evaluate the models in sentence realization for French and German and find that a particular conditional model outperforms all others. We employ a version of that model in an evaluation on unordered trees from the Penn TreeBank. We offer this result on standard data as a reference-point for evaluations of ordering in sentence realization. 1 Introduction Word and constituent order play a crucial role in establishing the fluency and intelligibility of a sentence. In some systems, establishing order during the sentence realization stage of natural language generation has been accomplished by hand-crafted generation grammars in the past (see for example, Aikawa et al. (2001) and Reiter and Dale (2000)). In contrast, the Nitrogen (Langkilde and Knight, 1998a, 1998b) system employs a word n-gram language model to choose among a large set of word sequence candidates which vary in constituent order, word order, lexical choice, and morphological inflection. Nitrogen’s model does not take into consideration any non-surface linguistic features available during realization. The Fergus system (Bangalore and Rambow, 2000) employs a statistical tree model to select probable trees and a word n-gram model to rank the string candidates generated from the best trees. Like Nitrogen, the HALogen system (Langkilde, 2000; Langkilde-Geary, 2002a, 2002b) uses word n-grams, but it extracts the best-scoring surface realizations efficiently from a packed forest by constraining the search first within the scope of each constituent. Our research is carried out within the Amalgam broad coverage sentence realization system. Amalgam generates sentence strings from abstract predicate-argument structures (Figure 1), using a pipeline of stages, many of which employ machine-learned models to predict where to perform specific linguistic operations based on the linguistic context (Corston-Oliver et al., 2002; Gamon et al., 2002a, 2002b; Smets et al., 2003). Amalgam has an explicit ordering stage that determines the order of constituents and their daughters. The input for this stage is an unordered tree of constituents; the output is an ordered tree of constituents or a ranked list of such trees. For ordering, Amalgam leverages tree constituent structure and, importantly, features of those constituents and the surrounding context. By separately establishing order within constituents, Amalgam heavily constrains the possible alternatives in later stages of the realization process. The design allows for interaction between ordering choices and other realization decisions, such as lexical choice (not considered in the present work), through score combinations from distinct …

Content truncated to 500 words