RMM: A Recursive Mental Model for Dialogue Navigation

Findings of the Association for Computational Linguistics: EMNLP 2020, pages 1732–1745 November 16 - 20, 2020. c⃝2020 Association for Computational Linguistics 1732 RMM: A Recursive Mental Model for Dialogue Navigation Homero Roman Roman1 Yonatan Bisk1,2 Jesse Thomason3 Asli Celikyilmaz1 Jianfeng Gao1 1Microsoft Research 2Carnegie Mellon University 3University of Washington horomanr@microsoft.com ybisk@cs.cmu.edu Abstract Language-guided robots must be able to both ask humans questions and understand answers. Much existing work focuses only on the latter. In this paper, we go beyond instruction follow- ing and introduce a two-agent task where one agent navigates and asks questions that a sec- ond, guiding agent answers. Inspired by the- ory of mind, we propose the Recursive Men- tal Model (RMM). The navigating agent models the guiding agent to simulate answers given candidate generated questions. The guiding agent in turn models the navigating agent to simulate navigation steps it would take to gen- erate answers. We use the progress agents make towards the goal as a reinforcement learning reward signal to directly inform not only navigation actions, but also both question and answer generation. We demonstrate that RMM enables better generalization to novel en- vironments. Interlocutor modelling may be a way forward for human-agent dialogue where robots need to both ask and answer questions. 1 Introduction A key challenge for embodied language is moving beyond instruction following to instruction genera- tion, which can require understanding the listener. The turn-based dialogue paradigm raises a myriad of new research questions, from grounded versions of traditional problems like co-reference resolution (Das et al., 2017a) to explicitly modeling theory of mind in order to consider the listener’s ability to understand generated instructions (Bisk et al., 2020). In this paper, we develop end-to-end di- alogue agents to navigate photorealistic, indoor scenes to reach goal rooms. We train agents us- ing the human-human Collaborative Vision-and- Dialogue Navigation (CVDN) (Thomason et al., 2019) dataset. CVDN dialogues are turn-based, with a navigator following guide instructions and asking questions when needed. RMM Humans Baseline Should I head forward? RMM’’ RMM’ RMM Yes, all the way down the hall Should I turn left here? … Figure 1: The RMM agent recursively models conversa- tions with instances of itself to choose the right ques- tions to ask (and answers to give) to reach the goal. Modeling turn-based dialogues involves four core challenges: C1 A navigator deciding when to ask a question. C2 Generating navigator questions. C3 Generating guide question answers. C4 Generating navigator actions. Prior work has addressed individual components of turn-based dialogue modeling. This work is the first to train navigator and guide agents to perform end-to-end, collaborative dialogues with question generation (C2), question answering (C3), and nav- igation (C4) conditioned on dialogue history. Theory of mind (Gopnik and Wellman, 1992) posits that efficient questions and answers build on a shared world of experiences and referents. To communicate efficiently, people model both a lis- tener’s mental state and the effects of their actions on the world. Modeling future worlds in naviga- tion (Anderson et al., 2019) …

Content truncated to 500 words