Exploiting Multi-Word Units in History-Based Probabilistic Generation

Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pp. 267–276, Prague, June 2007. c⃝2007 Association for Computational Linguistics Exploiting Multi-Word Units in History-Based Probabilistic Generation Deirdre Hogan, Conor Cafferkey, Aoife Cahill∗and Josef van Genabith National Centre for Language Technology School of Computing, Dublin City University Dublin 9, Ireland dhogan,ccafferkey,josef@computing.dcu.ie Abstract We present a simple history-based model for sentence generation from LFG f-structures, which improves on the accuracy of previous models by breaking down PCFG indepen- dence assumptions so that more f-structure conditioning context is used in the predic- tion of grammar rule expansions. In addi- tion, we present work on experiments with named entities and other multi-word units, showing a statistically significant improve- ment of generation accuracy. Tested on sec- tion 23 of the Penn Wall Street Journal Tree- bank, the techniques described in this paper improve BLEU scores from 66.52 to 68.82, and coverage from 98.18% to 99.96%. 1 Introduction Sentence generation, or surface realisation, is the task of generating meaningful, grammatically cor- rect and fluent text from some abstract semantic or syntactic representation of the sentence. It is an im- portant and growing field of natural language pro- cessing with applications in areas such as transfer- based machine translation (Riezler and Maxwell, 2006) and sentence condensation (Riezler et al., 2003). While recent work on generation in restricted domains, such as (Belz, 2007), has shown promising results there remains much room for improvement particularly for broad coverage and robust genera- tors, like those of Nakanishi et al. (2005) and Cahill ∗ Now at the Institut f¨ur Maschinelle Sprachverarbeitung, Universit¨at Stuttgart, Azenbergstrae 12, D-70174 Stuttgart, Germany. aoife.cahill@ims.uni-stuttgart.de and van Genabith (2006), which do not rely on hand- crafted grammars and thus can easily be ported to new languages. This paper is concerned with sentence genera- tion from Lexical-Functional Grammar (LFG) f- structures (Kaplan, 1995). We present improve- ments in previous LFG-based generation models firstly by breaking down PCFG independence as- sumptions so that more f-structure conditioning con- text is included when predicting grammar rule ex- pansions. This history-based approach has worked well in parsing (Collins, 1999; Charniak, 2000) and we show that it also improves PCFG-based genera- tion. We also present work on utilising named entities and other multi-word units to improve generation results for both accuracy and coverage. There has been a limited amount of exploration into the use of multi-word units in probabilistic parsing, for ex- ample in (Kaplan and King, 2003) (LFG parsing) and (Nivre and Nilsson, 2004) (dependency pars- ing). We are not aware of any similar work on gen- eration. In the LFG-based generation algorithm pre- sented by Cahill and van Genabith (2006) complex named entities (i.e. those consisting of more than one word token) and other multi-word units can be fragmented in the surface realization. We show that the identification of such units may be used as a sim- ple measure to constrain the generation model’s out- put. We take the generator of (Cahill …

Content truncated to 500 words