Empirically Estimating Order Constraints for Content Planning in Generation

Empirically Estimating Order Constraints for Content Planning in Generation Pablo A. Duboue and Kathleen R. McKeown Computer Science Department Columbia University 10027, New York, NY, USA {pablo,kathy}@cs.columbia.edu Abstract In a language generation system, a content planner embodies one or more “plans” that are usually hand–crafted, sometimes through manual analysis of target text. In this paper, we present a system that we developed to automati- cally learn elements of a plan and the ordering constraints among them. As training data, we use semantically an- notated transcripts of domain experts performing the task our system is de- signed to mimic. Given the large degree of variation in the spoken language of the transcripts, we developed a novel al- gorithm to find parallels between tran- scripts based on techniques used in computational genomics. Our proposed methodology was evaluated two–fold: the learning and generalization capabil- ities were quantitatively evaluated us- ing cross validation obtaining a level of accuracy of 89%. A qualitative evalua- tion is also provided. 1 Introduction In a language generation system, a content plan- ner typically uses one or more “plans” to rep- resent the content to be included in the out- put and the ordering between content elements. Some researchers rely on generic planners (e.g., (Dale, 1988)) for this task, while others use plans based on Rhetorical Structure Theory (RST) (e.g., (Bouayad-Aga et al., 2000; Moore and Paris, 1993; Hovy, 1993)) or schemas (e.g., (McKe- own, 1985; McKeown et al., 1997)). In all cases, constraints on application of rules (e.g., plan op- erators), which determine content and order, are usually hand-crafted, sometimes through manual analysis of target text. In this paper, we present a method for learn- ing the basic patterns contained within a plan and the ordering among them. As training data, we use semantically tagged transcripts of domain ex- perts performing the task our system is designed to mimic, an oral briefing of patient status af- ter undergoing coronary bypass surgery. Given that our target output is spoken language, there is some level of variability between individual tran- scripts. It is difficult for a human to see patterns in the data and thus supervised learning based on hand-tagged training sets can not be applied. We need a learning algorithm that can discover order- ing patterns in apparently unordered input. We based our unsupervised learning algorithm on techniques used in computational genomics (Durbin et al., 1998), where from large amounts of seemingly unorganized genetic sequences, pat- terns representing meaningful biological features are discovered. In our application, a transcript is the equivalent of a sequence and we are searching for patterns that occur repeatedly across multiple sequences. We can think of these patterns as the basic elements of a plan, representing small clus- ters of semantic units that are similar in size, for example, to the nucleus-satellite pairs of RST.1 By learning ordering constraints over these ele- tion.1Note, however, that we do not learn or represent inten- age, gender, pmh, pmh, pmh, pmh, med-preop, med-preop, med-preop, drip-preop, med-preop, ekg-preop, echo-preop, …

Content truncated to 500 words