A Textual Dataset for Situated Proactive Response Selection

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 3856–3874 July 9-14, 2023 ©2023 Association for Computational Linguistics A Textual Dataset for Situated Proactive Response Selection Naoki Otaniα∗, Jun Arakiβ, HyeongSik Kimβ, Eduard Hovyγ αMegagon Labs, Mountain View, CA, USA βRobert Bosch LLC, Sunnyvale, CA, USA γUniversity of Melbourne, Melbourne, VIC, Australia αnotani@alumni.cmu.edu β{jun.araki,hyeongsik.kim}@us.bosch.com γeduard.hovy@unimelb.edu.au Abstract Recent data-driven conversational models are able to return fluent, consistent, and informa- tive responses to many kinds of requests and utterances in task-oriented scenarios. However, these responses are typically limited to just the immediate local topic instead of being wider- ranging and proactively taking the conversa- tion further, for example making suggestions to help customers achieve their goals. This in- adequacy reflects a lack of understanding of the interlocutor’s situation and implicit goal. To address the problem, we introduce a task of proactive response selection based on situ- ational information. We present a manually- curated dataset of 1.7k English conversation examples that include situational background information plus for each conversation a set of responses, only some of which are acceptable in the situation. A responsive and informed conversation system should select the appro- priate responses and avoid inappropriate ones; doing so demonstrates the ability to adequately understand the initiating request and situation. Our benchmark experiments show that this is not an easy task even for strong neural models, offering opportunities for future research. 1 Introduction Conversational assistant systems have recently shown significant improvements for understanding users’ inquiries along with background knowledge, conducting requested operations, and returning nat- ural language responses. Yet, typical systems are likely to be passive and only process user-initiated requests or merely ask values for domain-specific slots (Williams et al., 2013; Ammari et al., 2019). In contrast, human assistants like hotel concierges are more proactive, acting to address unmentioned needs and expected future events (Cho et al., 1996; Bellini and Convert, 2016). They do not only make ∗This work was done while the first author was at Carnegie Mellon University. The room temperature is hot. Situation 1 Can you open the window? User utterance The room is smoky. Situation 2 Candidate A Sure. Shall I run the air purifier, too? To cool off The user’s Implicit goal To air the room System response Candidate B Sure. Shall I bring you cold water, too? Figure 1: An example of situated goal-aware proactive response selection. The response candidate A is appro- priate in Situation 1 but not in Situation 2. a direct response or a clarification question to their interlocutors but also provide personalized infor- mation/assistance based on context and knowledge. To push the frontier of task-oriented conversa- tion technologies, we propose a task of proactive response selection for single-turn help-seeking con- versations in English. We mean by proactive that a system engages in an interaction in a coopera- tive manner (Grice, 1975) and suggests something helpful to a user. The proposed task touches upon two crucial aspects of help-seeking conversations: situation-awareness and goal-awareness. Situation: …

Content truncated to 500 words