Help, Anna! Visual Navigation with Natural Multimodal Assistance via Retrospective Curiosity-Encouraging Imitation Learning

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 684–695, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 684 Help, Anna! Visual Navigation with Natural Multimodal Assistance via Retrospective Curiosity-Encouraging Imitation Learning Khanh Nguyen⊙ and Hal Daum´e III⊙♥ University of Maryland, College Park⊙, Microsoft Research, New York♥ {kxnguyen,hal}@umiacs.umd.edu Abstract Mobile agents that can leverage help from hu- mans can potentially accomplish more com- plex tasks than they could entirely on their own. We develop “Help, Anna!” (HANNA), an interactive photo-realistic simulator in which an agent fulfills object-finding tasks by re- questing and interpreting natural language- and-vision assistance. An agent solving tasks in a HANNA environment can leverage simu- lated human assistants, called ANNA (Auto- matic Natural Navigation Assistants), which, upon request, provide natural language and vi- sual instructions to direct the agent towards the goals. To address the HANNA problem, we develop a memory-augmented neural agent that hierarchically models multiple levels of decision-making, and an imitation learning al- gorithm that teaches the agent to avoid re- peating past mistakes while simultaneously predicting its own chances of making future progress. Empirically, our approach is able to ask for help more effectively than competitive baselines and, thus, attains higher task suc- cess rate on both previously seen and previ- ously unseen environments. We publicly re- lease code and data at https://github. com/khanhptnk/hanna . 1 Introduction The richness and generalizability of natural lan- guage makes it an effective medium for directing mobile agents in navigation tasks, even in environ- ments they have never encountered before (Ander- son et al., 2018b; Chen et al., 2019; Misra et al., 2018; de Vries et al., 2018; Qi et al., 2019). Nev- ertheless, even with language-based instructions, such tasks can be overly difficult for agents on their own, especially in unknown environments. To accomplish tasks that surpass their knowl- edge and skill levels, agents must be able to ac- tively seek for and leverage assistance in the en- vironment. Humans are rich external knowledge sources but, unfortunately, they may not be avail- able all the time to provide guidance, or may be unwilling to help too frequently. To reduce the needed effort from human assistants, it is essential to design research platforms for teaching agents to request help mindfully. In natural settings, human assistance is often: ⋄ derived from interpersonal interaction (a lost tourist asks a local for directions); ⋄ reactive to the situation of the receiver, based on the assistant’s knowledge (the local may guide the tourist to the goal, or may redirect them to a different source of assistance); ⋄ delivered via a multimodal communication channel (the local uses a combination of lan- guage, images, maps, gestures, etc.). We introduce the “Help, Anna!” (HANNA) prob- lem (§ 3), in which a mobile agent has to navi- gate (without a map) to an object by interpreting its first-person visual perception and requesting help from Automatic Natural Navigation Assis- tants …

Content truncated to 500 words