RUN through the Streets: A New Dataset and Baseline Models for Realistic Urban Navigation

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 6449–6455, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 6449 RUN through the Streets: A New Dataset and Baseline Models for Realistic Urban Navigation Tzuf Paz-Argaman1 and Reut Tsarfaty1,2 1Open University of Israel 2Allen Institute for Artificial Intelligence {tzufar,reut.tsarfaty}@gmail.com Abstract Following navigation instructions in natural language requires a composition of language, action, and knowledge of the environment. Knowledge of the environment may be pro- vided via visual sensors or as a symbolic world representation referred to as a map. Here we introduce the Realistic Urban Navigation (RUN) task, aimed at interpreting navigation instructions based on a real, dense, urban map. Using Amazon Mechanical Turk, we collected a dataset of 2515 instructions aligned with ac- tual routes over three regions of Manhattan. We propose a strong baseline for the task and empirically investigate which aspects of the neural architecture are important for the RUN success. Our results empirically show that entity abstraction, attention over words and worlds, and a constantly updating world-state, significantly contribute to task accuracy. 1 Introduction and Background The task of interpreting and following natural lan- guage (NL) navigation instructions involves in- terleaving different signals, at the very least the linguistic utterance and the representation of the world. For example, in “turn right on the first in- tersection”, the instruction needs to be interpreted, and a specific object in the world (the intersec- tion) needs to be located in order to execute the instruction. In NL navigation studies, the repre- sentation of the world may be provided via visual sensors (Misra et al., 2018; Blukis et al., 2018; Nguyen et al., 2018; Yan et al., 2018; Anderson et al., 2018) or as a symbolic world representation. This work focuses on navigation based on a sym- bolic world representation (referred to as a map). Previous datasets for NL navigation based on a symbolic world representation, HCRC (Anderson et al., 1991; Vogel and Jurafsky, 2010; Levit and Roy, 2007) and SAIL (MacMahon et al., 2006; Chen and Mooney, 2011; Kim and Mooney, 2012, 2013; Artzi and Zettlemoyer, 2013; Artzi et al., 2014; Fried et al., 2017; Andreas and Klein, 2015) present relatively simple worlds, with a small fixed set of entities known to the navigator in advance. Such representations bypass the great complexity of real urban navigation, which consists of long paths and an abundance of previously unseen enti- ties of different types. In this work we introduce Realistic Urban Nav- igation (RUN), where we aim to interpret naviga- tion instructions relative to a rich symbolic rep- resentation of the world, given by a real dense urban map. To address RUN, we designed and collected a new dataset based on OpenStreetMap, in which we align NL instructions to their corre- sponding routes. Using Amazon Mechanical Turk, we collected 2515 instructions over 3 regions of Manhattan, all specified (and verified) by (respec- tive) sets of …

Content truncated to 500 words