Continual Adaptation for Efficient Machine Communication

Proceedings of the 24th Conference on Computational Natural Language Learning, pages 408–419 Online, November 19-20, 2020. c⃝2020 Association for Computational Linguistics https://doi.org/10.18653/v1/P17 408 Continual Adaptation for Efficient Machine Communication Robert D. Hawkins1, Minae Kwon2, Dorsa Sadigh2,3, Noah D. Goodman1,2 Departments of 1Psychology, 2Computer Science, and 3Electrical Engineering Stanford University, Stanford, CA 94305 {rxdh, mnkwon, ngoodman}@stanford.edu, dorsa@cs.stanford.edu Abstract To communicate with new partners in new con- texts, humans rapidly form new linguistic con- ventions. Recent neural language models are able to comprehend and produce the existing conventions present in their training data, but are not able to flexibly and interactively adapt those conventions on the fly as humans do. We introduce an interactive repeated reference task as a benchmark for models of adaptation in communication and propose a regularized continual learning framework that allows an artificial agent initialized with a generic lan- guage model to more accurately and efficiently communicate with a partner over time. We evaluate this framework through simulations on COCO and in real-time reference game ex- periments with human partners. 1 Introduction Communication depends on shared conventions about the meanings of words (Lewis, 1969), but the real-world demands of language use often re- quire agents to go beyond fixed conventional mean- ings (Grice, 1975; Davidson, 1986). Recent work on pragmatic and context-aware models has ap- proached this problem by equipping speaker and listener agents with the ability to explicitly rea- son about one another. Pragmatic reasoning al- lows listeners to infer richer intended meanings by considering counterfactual alternatives, and al- lows speakers to be appropriately informative, not merely truthful (Goodman and Frank, 2016; An- dreas and Klein, 2016; Fried et al., 2018; Monroe et al., 2017; Vedantam et al., 2017). These models have largely focused on one-shot settings, where the context is the immediate visual environment. In common interactive settings, how- ever, the relevant context for pragmatic competence also includes the history of previous interactions with the same communication partner. Human in- a cat lying in the sunlight on a bed two cats are playing on a log trial 1 trial 2 ... initialization ... Figure 1: We introduce a regularized continual learn- ing approach allowing agents initialized with a pre- trained language model Θ to iteratively infer the lan- guage model θi used by a partner, over repeated inter- actions {t1, t2 . . . } in an online reference game. terlocutors are able to establish ad hoc conventions based on this history (Clark and Wilkes-Gibbs, 1986; Clark, 1996), allowing for increasingly accu- rate and efficient communication. Speakers can re- main understandable while expending significantly fewer words (Krauss and Weinheimer, 1964; Orita et al., 2015; Stali¯unait˙e et al., 2018; Hawkins et al., 2020a; Stewart et al., 2020). For example, consider a nurse visiting a bed- ridden patient at their home. The first time the patient asks the nurse to retrieve a particular medi- cation, they must painstakingly identify a specific bottle, e.g. “the medicine for my back pain in a small blue medicine bottle labeled Flexeril in …

Content truncated to 500 words