Generating Spatial Descriptions for Cross-modal References

Generating Spatial Descriptions for Cross-modal References Peter Wazinski SFB 314, Department of Computer Science University of Saarbrficken D-6600 Saarbrficken, Germany email: wazinski@cs.uni-sb.de Abstract We present a localisation component that sup- ports the generation of cross-modal deictic ex- pressions in the knowledge-based presentation system WIP. We deal with relative localisations (e.g., "The object to the left, of object X."), absolute localisations (e.g., "The object in the upl)er left part of the l)icture.") and corner lo- calisations (e.g., "The object in the lower right corner of the l)icture"). In addition, we distin- guish two localisation granularities, one less de- tailed (e.g., "the object to the left. of object X.") and one more detailed (e.g., "the object above and to the left. of object X."). We consider cor- ner localisations to be similar to absolute local- isations and in turn absolute localisations to be specialisations of relative localisations. This al- lows us to compute all three localisation types with one generic localisation procedure. As elementary localisations are derived from pre- viously computed composite localisations, we can cope with both localisation granularities in a computationally efficient way. Based on these l)rimary localisation l)rocedures, we dis- cuss how objects can be localised among several other objects. Finally we introduce group local- isations (e.g., "The object to left, of the group of or, her objects.") and show how to deal with thern. 1 Introduction The increasing amount of information to be communi- cated to users of complex technical systems nowadays makes it necessary to find new ways to present infor- mation. Neither the variety of all possible l)resentation situations can be anticipated nor it is fiLrther adequate to present the required information in a single communi- cation mode, such as either text or graphics. Therefore, the automatic generation of nmltimodal presentations tailored to the individual user has become necessary. Current research projects in artificial intelligence like SAGE ([Roth et al., 1990]), FN/ANDD ([Marks and Reiter, 1990]), COMET ([Feiner and MeKeown, 1990]) and WIP ([Wahlster el al., 1991a]) reflect the growing interest in this topic. For the knowledge-based presentation system WIP, the task is the generation of a multimodal document ac- cording to the formal description of the communicative intent of the planned presentation and a set of generation parameters. The current scenario for WIP is the gener- ation of instructions for using an espresso-machine. A typical fragment of an instruction manual for an espresso machine is shown in figure 1. b I A B C ©®© Before you lift the lid make sure that the knob in the middle is in position C. \o \ ®¢® Remove the cover and pour in cold tap water. Figure 1: Fragment from an instruction manual Cross-modal deictic expressions, e.g., "the lid" or "the knob in the middle," help to establish the coreferentiality between the entities mentioned in the text and shown in the picture as well ([Wahlster et al., 1991b]). The use o! spatial relationships such as "the knob in the middle" simplifies the generation of referring expressions that …

Content truncated to 500 words