Probabilistic Generation of Weather Forecast Texts

Proceedings of NAACL HLT 2007, pages 164–171, Rochester, NY, April 2007. c⃝2007 Association for Computational Linguistics Probabilistic Generation of Weather Forecast Texts Anja Belz Natural Language Technology Group University of Brighton, UK a.s.belz@brighton.ac.uk Abstract This paper reports experiments which pCRU — a generation frameworkin that combines probabilistic generation methodology with a comprehensive model of the generation space — is used to semi-automatically create sev- eral versions of a weather forecast text generator. The generators are evaluated in terms of output quality, development time and computational efficiency against (i) human forecasters, (ii) a traditional handcrafted pipelined NLG system, and (iii) a HALOGEN-style statistical genera- tor. The most striking result is that despite acquiring all decision-making abilities automatically, the best pCRU generators receive higher scores from human judges than forecasts written by experts. 1 Introduction and background Over the last decade, there has been a lot of in- terest in statistical techniques among researchers in natural language generation (NLG), a field that was largely unaffected by the statistical revolution in NLP that started in the 1980s. Since Langkilde and Knight’s influential work on statistical surface real- isation (Knight and Langkilde, 1998), a number of statistical and corpus-based methods have been re- ported. However, this interest does not appear to have translated into practice: of the 30 implemented systems and modules with development starting in or after 2000 that are listed on a key NLG website1, only five have any statistical component at all (an- other six involve techniques that are in some way corpus-based). The likely reasons for this lack of take-up are that (i) many existing statistical NLG techniques are inherently expensive, requiring the set of alternatives to be generated in full before the statistical model is applied to select the most likely; and (ii) statistical NLG techniques have not been shown to produce outputs of high enough quality. There has also been a rethinking of the traditional modular NLG architecture (Reiter, 1994). Some re- search has moved towards a more comprehensive view, e.g. construing the generation task as a single constraint satisfaction problem. Precursors to cur- rent approaches were Hovy’s PAULINE which kept track of the satisfaction status of global ‘rhetori- cal goals’ (Hovy, 1988), and Power et al.’s ICON- OCLAST which allowed users to fine-tune different combinations of global constraints (Power, 2000). In recent comprehensive approaches, the focus is on automatic adaptability, e.g. automatically determin- ing degrees of constraint violability on the basis of corpus frequencies. Examples include Langkilde’s (2005) general approach to generation and parsing based on constraint optimisation, and Marciniak and Strube’s (2005) integrated, globally optimisable net- work of classifiers and constraints. Both probabilistic and recent comprehensive trends have developed at least in part to address two interrelated issues in NLG: the considerable amount 1Bateman and Zock’s list of NLG systems, http://www.fb10.uni-bremen.de/anglistik/ langpro/NLG-table/, 20/01/2006. 164 of time and expense involved in building new sys- tems, and the almost complete lack in the field of reusable systems and modules. Both trends have the potential to improve on development …

Content truncated to 500 words