I Cast Detect Thoughts: Learning to Converse and Guide with Intents and Theory-of-Mind in Dungeons and Dragons

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 11136–11155 July 9-14, 2023 ©2023 Association for Computational Linguistics I Cast Detect Thoughts: Learning to Converse and Guide with Intents and Theory-of-Mind in Dungeons and Dragons Pei Zhou♡♠ Andrew Zhu♣ Jennifer Hu † Jay Pujara♠ Xiang Ren♡♠ Chris Callison-Burch♣♡ Yejin Choi♢♡ Prithviraj Ammanabrolu♡♢ ♡ Allen Institute for Artificial Intelligence ♠ University of Southern California ♢ University of Washington ♣ University of Pennsylvania † MIT peiz@usc.edu raja@allenai.org Abstract We propose a novel task, G4C, to study teacher-student natural language interactions in a goal-driven and grounded environment. Dungeons and Dragons (D&D), a role-playing game, provides an ideal setting to investigate such interactions. Here, the Dungeon Master (DM), i.e., the teacher, guides the actions of several players—students, each with their own personas and abilities—to achieve shared goals grounded in a fantasy world. Our approach is to decompose and model these interactions into (1) the DM’s intent to guide players towards a given goal; (2) the DM’s guidance utterance to the players expressing this intent; and (3) a theory-of-mind (ToM) model that anticipates the players’ reaction to the guidance one turn into the future. We develop a novel reinforce- ment learning (RL) method for training a DM that generates guidance for players by reward- ing utterances where the intent matches the ToM-anticipated player actions. Human and automated evaluations show that a DM trained to explicitly model intents and incorporate ToM of the players using RL generates better-quality guidance that is 3x more likely to fulfill the DM’s intent than a vanilla natural language generation (NLG) approach. 1 Introduction Humans communicate with a goal in mind and use language to reach the goal by interacting with their communication partners grounded in a shared environment (Grice, 1975; Allwood, 1976; Clark and Schaefer, 1989; Clark and Brennan, 1991). To make sure the goal is reached, we often anticipate how the partners will respond in advance to steer the conversations in the desired direction. This abil- ity to reason about the mental states of conversation partners – i.e., theory-of-mind (ToM; Premack and Woodruff, 1978) – is key to smooth and efficient communication (Perner et al., 1989; Happé, 1993). Most existing dialogue agents, while able to pro- duce human-like responses, often do not model Inside DM's (Theory of) Mind DM Intent I want the players to make a perception check to find out about the goblins to get the letter "You notice some movements in the bushes" “There might be something hiding there, let’s go take a look. Perception check!” Players Anticipated Player Action Guidance Matches Intent Shared Common Ground between the DM and Players Players were hired by a dwarf named Gundren Rockseeker to transport a wagonload of provisions to Phandalin. After a day and a half of travel, the players got onto a smaller trail not as well maintained... Information Only Available to the DM Five goblins hid in the bushes near the trail ready to attack the players. Upon …

Content truncated to 500 words