Intelligent Multi Media Presentation of information in a semi-immersive Command and Control environment

Intelligent Multi Media Presentation of information in a semi-immersive Command and Control environment Cécile Paris, Nathalie Colineau Commonwealth Scientific and Industrial Research Organisation (CSIRO) ICT Centre Sydney, NSW, Australia Cecile.Paris@csiro.au Nathalie.Colineau@csiro.au Dominique Estival Defence Science & Technology Organisation (DSTO) Human Systems Integration Group Command and Control Division Edinburgh SA 5111, Australia Dominique.Estival@dsto.defence.gov.au Abstract We describe the framework for an intelligent multimedia presentation system we designed to be part of the FOCAL laboratory, a semi-immersive environment for Command and Control Environment. FOCAL comprises a number of input devices and output media, animated virtual conversational characters, a spoken dialogue system, and sophisticated visual displays. These need to be coordinated to provide a useful and effective presentation to the user. In this paper, we describe the principles which underlie intelligent multimedia presentation (IMMP) systems and the design of such a system within the FOCAL multi- agent architecture. 1 Introduction 1.1 Description of FOCAL FOCAL (Future Operations Centre Analysis Laboratory) was established at the Australian Defence Science and Technology Organisation (DSTO) to "pioneer a paradigm shift in command environments through a superior use of capability and greater situation awareness". The facility was designed to experiment with innovative technologies to support this goal, and it has now been running since 2000. FOCAL contains a large-screen, semi- immersive virtual reality environment, where large quantities of information can be displayed. A number of modalities and media are available to display the information to the end-user. These include visual display mechanisms, such as 3-D virtual batttlespace, and spoken dialogue interaction with virtual conversational characters (VCCs) that allow presentation of information through speech as well as through textual displays (Taplin et al. 2001; Broughton et al., 2002; Estival et al., 2003). While these have so far been studied and implemented somewhat independently of each other, ultimately all the different available means to present the information to the end-user must work together and be combined into a coherent whole; otherwise, the result would be very confusing to the user. From the delivery perspective (as opposed to the input fusion aspect) with which we are concerned here, FOCAL can be considered as an instance of an intelligent multimedia presentation (IMMP) system (see Bordegoni et al., 1997 for a reference model). 1.2 An IMMP Architecture for FOCAL The framework for the design of an intelligent multimedia presentation system (IMMP) within FOCAL was the result of a collaboration between DSTO and CSIRO. The aim was to design an architecture for the information delivery component, taking into account the existing architecture for the overall system, the available data sources and the type of desired presentations. One of the main idea in FOCAL is that a VCC will serve as a Virtual Adviser (VA) to the team of commanding officers engaged in the planning or conduct of an operation. The aim of the VA is to engage in interactions with the officers, presenting information and offering advice. VAs are able to present the information and justify their advice through multimedia presentations (e.g., speech, video, text, …

Content truncated to 500 words