Intensional Summaries as Cooperative Responses in Dialogue: Automation and Evaluation

Proceedings of ACL-08: HLT, pages 479–487, Columbus, Ohio, USA, June 2008. c⃝2008 Association for Computational Linguistics Intensional Summaries as Cooperative Responses in Dialogue: Automation and Evaluation Joseph Polifroni Nokia Research Center 3 Cambridge Center Cambridge, MA 02142, USA joseph.polifroni@nokia.com Marilyn Walker Department of Computer Science University of Sheffield Sheffield, S1 4DP, United Kingdom M.A.Walker@sheffield.ac.uk Abstract Despite its long history, and a great deal of research producing many useful algo- rithms and observations, research in coop- erative response generation has had little impact on the recent commercialization of dialogue technologies, particularly within the spoken dialogue community. We hy- pothesize that a particular type of cooper- ative response, intensional summaries, are effective for when users are unfamiliar with the domain. We evaluate this hypothe- sis with two experiments with cruiser, a DS for in-car or mobile users to access restaurant information. First, we compare cruiser with a baseline system-initiative DS, and show that users prefer cruiser. Then, we experiment with four algorithms for constructing intensional summaries in cruiser, and show that two summary types are equally effective: summaries that maximize domain coverage and summaries that maximize utility with respect to a user model. 1 Introduction Research in cooperative response generation has had a long history and produced many useful algorithms and observations (Mays 1980; Pol- lack et al.1982; Joshi et al., 1986; Kalita et al., 1986) inter alia. However, it has had little impact on the recent commercialization of dia- logue technologies, particularly within the spo- ken dialogue community. We believe that this lack of interest in cooperative response genera- tion arises from two limitations of previous work: (1) There has been relatively little empirical re- search showing that cooperative responses lead U1: Tell me about restaurants in London. SI1: What kind of cuisine are you interested in? C1: I know of 596 restaurants in London. I know of 3 inexpensive vegetarian restaurants and 14 inexpensive Chinese restaurants. I also know of 4 inexpensive Greek restaurants. U2: Chinese. SI2: Do you have a price range in mind? C2: I know of 27 restaurants in London that serve Chinese food. There are 8 inexpen- sive Chinese restaurants in Chinatown and 2 inexpensive Chinese restaurants in Hamp- stead/Kilburn. I also know of 1 inexpensive Chinese restaurant in Soho. U3: How about a cheap one? SI3: What neighborhood would you like? C3: I know of 1 inexpensive Chinese restaurant in Hampstead/Kilburn with very good food quality and 1 in Bayswater with good food quality. I also know of 2 in Chinatown with medium food quality. Figure 1: Intensional summaries (C = cruiser) as compared with a system initiative (SI) strategy in the London restaurant domain. U = User to more natural, effective, or efficient dialogues (Litman et al.1998; Demberg and Moore, 2006); and (2) Previous work has hand-crafted such re- sponses, or hand-annotated the database to sup- port them (Kaplan, 1984; Kalita et al., 1986; Cholvy, 1990; Polifroni et al., 2003; Benamara, 2004), which has made it difficult to port and scale these algorithms. Moreover, …

Content truncated to 500 words