Learning What to Talk about in Descriptive Games

Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing (HLT/EMNLP), pages 291–298, Vancouver, October 2005. c⃝2005 Association for Computational Linguistics Learning What to Talk About in Descriptive Games Hugo Zaragoza Microsoft Research Cambridge, United Kingdom hugoz@microsoft.com Chi-Ho Li University of Sussex Brighton, United Kingdom C.H.Li@sussex.ac.uk Abstract Text generation requires a planning mod- ule to select an object of discourse and its properties. This is specially hard in de- scriptive games, where a computer agent tries to describe some aspects of a game world. We propose to formalize this prob- lem as a Markov Decision Process, in which an optimal message policy can be defined and learned through simulation. Furthermore, we propose back-off poli- cies as a novel and effective technique to fight state dimensionality explosion in this framework. 1 Introduction Traditionally, text generation systems are decom- posed into three modules: the application module which manages the high-level task representation (state information, actions, goals, etc.), the text plan- ning module which chooses messages based on the state of the application module, and the sentence generation module which transforms messages into sentences. The planning module greatly depends on the characteristics of both the application and the generation modules, solving issues in domain modelling, discourse and sentence planning, and to some degree lexical and feature selection (Cole et al., 1997). In this paper we concentrate on one of the most basic tasks that text planning needs to solve: selecting the message content, or more sim- ply, choosing what to talk about. Work on text-generation often assumes that an object or topic has been already chosen for discus- sion. This is reasonable for many applications, but in some cases choosing what to talk about can be harder than choosing how to. This is the case in the type of text generation applications that we are in- terested in: generating descriptive messages in com- puter games. In a modern computer game at any given moment there may be an enormous number of object properties that can be described, each with varying importance and consequences. The outcome of the game depends not only on the skill of the player, but also on the quality of the descriptive mes- sages produced. We refer to such situations as de- scriptive games. Our goal is to develop a strategy to choose the most interesting descriptive messages that a particu- lar talker may communicate to a particular listener, given their context (i.e. their knowledge of the world and of each-other). We refer to this as message plan- ning. Developing a general framework for planning is very difficult because of the strong coupling be- tween the planning and application modules. We propose to frame message planning as a Markov De- cision Process (MDP) which encodes the environ- ment, the information available to the talker and lis- tener, the consequences of their communicative and non-communicative acts, and the constraints of the text generation module. Furthermore we propose to use Reinforcement Learning (RL) to learn the op- …

Content truncated to 500 words