Empirically-based Control of Natural Language Generation

Proceedings of the 43rd Annual Meeting of the ACL, pages 58–65, Ann Arbor, June 2005. c⃝2005 Association for Computational Linguistics Empirically-based Control of Natural Language Generation Daniel S. Paiva Roger Evans Department of Informatics Information Technology Research Institute University of Sussex University of Brighton Brighton, UK Brighton, UK danielpa@sussex.ac.uk Roger.Evans@itri.brighton.ac.uk Abstract In this paper we present a new approach to controlling the behaviour of a natural lan- guage generation system by correlating in- ternal decisions taken during free generation of a wide range of texts with the surface sty- listic characteristics of the resulting outputs, and using the correlation to control the gen- erator. This contrasts with the generate-and- test architecture adopted by most previous empirically-based generation approaches, offering a more efficient, generic and holis- tic method of generator control. We illus- trate the approach by describing a system in which stylistic variation (in the sense of Biber (1988)) can be effectively controlled during the generation of short medical in- formation texts. 1 Introduction This paper1 is concerned with the problem of con- trolling the output of natural language generation (NLG) systems. In many application scenarios the generator’s task is underspecified, resulting in mul- tiple possible solutions (texts expressing the de- sired content), all equally good to the generator, but not equally appropriate for the application. Customising the generator directly to overcome this generally leads to ad-hoc, non-reusable solu- tions. A more modular approach is a generate-and- test architecture, in which all solutions are gener- ated, and then ranked or otherwise selected accord- ing to their appropriateness in a separate post- 1 Paiva and Evans (2004) provides an overview of our framework and detailed comparison with previous approaches to stylistic control (like Hovy (1988), Green and DiMarco (1993) and Langkilde-Geary (2002)). This paper provides a more detailed account of the system and reports additional experimental re- sults. process. Such architectures have been particularly prominent in the recent development of empiri- cally-based approaches to NLG, where generator outputs can be selected according to application requirements acquired directly from human sub- jects (e.g. Walker et al. (2002)) or statistically from a corpus (e.g. Langkilde-Geary (2002)). However, this approach suffers from a number of drawbacks: 1. It requires generation of all, or at least many solutions (often hundreds of thou- sands), expensive both in time and space, and liable to lead to unnecessary interac- tions with other components (e.g. knowl- edge bases) in complex systems. Recent advances in the use of packed representa- tions ameliorate some of these issues, but the basic need to compare a large number of solutions in order to rank them remains. 2. The ‘test’ component generally does not give fine-grained control — for example, in a statistically-based system it typically measures how close a text is to some sin- gle notion of ideal (actually, statistically average) output. 3. Use of an external filter does not combine well with any control mechanisms within the generator: e.g. controlling combinato- rial explosion of modifier attachment or adjective order. In this paper …

Content truncated to 500 words