ENGAGE: Automated Gestures for Animated Characters

ENGAGE: Automated Gestures for Animated Characters Marcin Nowina-Krowicki, Andrew Zschorn, Michael Pilling and Steven Wark Command, Control, Communications and Intelligence Division, Defence Science and Technology Organisation, Edinburgh, South Australia {firstname.lastname}@dsto.defence.gov.au Abstract There is a rapidly growing body of work in the use of Embodied Conversational Agents (ECA) to convey complex con- textual relationships through verbal and non-verbal communication, in domains ranging from military C2 to e-learning. In these applications the subject matter ex- pert is often naïve to the technical re- quirements of ECAs. ENGAGE (the Ex- tensible Natural Gesture Animation Gen- eration Engine) is designed to automati- cally generate appropriate and ‘realistic’ animation for ECAs based on the content provided to them. It employs syntactic analysis of the surface text and uses pre- defined behaviour models to generate ap- propriate behaviours for the ECA. We discuss the design of this system, its cur- rent applications and plans for its future development. 1 Introduction The Defence Science and Technology Organisa- tion has an active research program into the use of multimedia narrative to provide situational awareness for military C2 (Wark and Lambert 2007). In common usage, face-to-face communi- cation is the predominant, and often most effec- tive, way for people to give and obtain complex contextual information. Embodied Conversa- tional Agents (ECA) provide verbal and non- verbal communication modes similar to face-to- face communication. Gestures such as nods and facial expressions are very important in listener engagement with the speaker and their message. Programming these gestures into an ECA ani- mation is time consuming and requires special- ised expertise. The subject matter experts devel- oping content for ECAs are often naïve with re- spect to these technical requirements. A system to automatically generate appropriate non-verbal behaviour allows the content creator to concen- trate on the information and not on how the ECA will animate it. The BEAT system from MIT (Cassell et al. 2001) demonstrated this capability. DSTO has developed ENGAGE (Extensible Natural Ges- ture Animation Generation Engine) based on the principles demonstrated in BEAT, and extended them to incorporate modifiers such as confi- dence, importance, and urgency. 1.1 Virtual Adviser DSTO has been using ECAs dubbed Virtual Ad- visers (VAs) as a mechanism for augmenting situational awareness in military C2 (Taplin et al. 2001; Wark and Lambert 2007; Wark et al. 2009). Virtual Advisers are computer generated talking heads using photo realistic textures with real-time animation and commercial-off-the-shelf text-to-speech (TTS) generation. Virtual Advis- ers can also include rolling text captions and multimedia monitors à la television news ser- Figure 1 – Virtual Advisers present photo-realistic models of people Marcin Nowina-Krowicki, Andrew Zschorn, Michael Pilling and Steven Wark. 2011. ENGAGE: Automated Gestures for Animated Characters. In Proceedings of Australasian Language Technology Association Workshop, pages 166−174 vices. Virtual Advisers have been designed for modularity and can be delivered to users in a number of ways. VAs are used to present situation briefs incor- porating other media such as tables and dia- grams, images, video, 3D models and so on. They are …

Content truncated to 500 words