Learning to Adapt to Unknown Users: Referring Expression Generation in Spoken Dialogue Systems

Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 69–78, Uppsala, Sweden, 11-16 July 2010. c⃝2010 Association for Computational Linguistics Learning to Adapt to Unknown Users: Referring Expression Generation in Spoken Dialogue Systems Srinivasan Janarthanam School of Informatics University of Edinburgh s.janarthanam@ed.ac.uk Oliver Lemon Interaction Lab Mathematics and Computer Science (MACS) Heriot-Watt University o.lemon@hw.ac.uk Abstract We present a data-driven approach to learn user-adaptive referring expression gener- ation (REG) policies for spoken dialogue systems. Referring expressions can be dif- ficult to understand in technical domains where users may not know the techni- cal ‘jargon’ names of the domain entities. In such cases, dialogue systems must be able to model the user’s (lexical) domain knowledge and use appropriate referring expressions. We present a reinforcement learning (RL) framework in which the sys- tem learns REG policies which can adapt to unknown users online. Furthermore, unlike supervised learning methods which require a large corpus of expert adaptive behaviour to train on, we show that effec- tive adaptive policies can be learned from a small dialogue corpus of non-adaptive human-machine interaction, by using a RL framework and a statistical user simula- tion. We show that in comparison to adaptive hand-coded baseline policies, the learned policy performs significantly bet- ter, with an 18.6% average increase in adaptation accuracy. The best learned pol- icy also takes less dialogue time (average 1.07 min less) than the best hand-coded policy. This is because the learned poli- cies can adapt online to changing evidence about the user’s domain expertise. 1 Introduction We present a reinforcement learning (Sutton and Barto, 1998) framework to learn user-adaptive re- ferring expression generation policies from data- driven user simulations. A user-adaptive REG pol- icy allows the system to choose appropriate ex- pressions to refer to domain entities in a dialogue Jargon: Please plug one end of the broadband cable into the broadband filter. Descriptive: Please plug one end of the thin white cable with grey ends into the small white box. Table 1: Referring expression examples for 2 enti- ties (from the corpus) setting. For instance, in a technical support con- versation, the system could choose to use more technical terms with an expert user, or to use more descriptive and general expressions with novice users, and a mix of the two with intermediate users of various sorts (see examples in Table 1). In natural human-human conversations, dia- logue partners learn about each other and adapt their language to suit their domain expertise (Is- sacs and Clark, 1987). This kind of adaptation is called Alignment through Audience Design (Clark and Murphy, 1982; Bell, 1984). We assume that users are mostly unknown to the system and therefore that a spoken dialogue system (SDS) must be capable of observing the user’s dialogue behaviour, modelling his/her do- main knowledge, and adapting accordingly, just like human interlocutors. Rule-based and super- vised learning approaches to user adaptation in SDS have been proposed earlier (Cawsey, 1993; Akiba and Tanaka, 1994). However, such methods require expensive resources such …

Content truncated to 500 words