Multi-agent Communication meets Natural Language: Synergies between Functional and Structural Language Learning

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7663–7674 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 7663 Multi-agent Communication meets Natural Language: Synergies between Functional and Structural Language Learning Angeliki Lazaridou∗, Anna Potapenko∗, Olivier Tieleman∗ DeepMind, London, UK {angeliki,apotapenko,tieleman}@google.com Abstract We present a method for combining multi- agent communication and traditional data- driven approaches to natural language learn- ing, with an end goal of teaching agents to communicate with humans in natural language. Our starting point is a language model that has been trained on generic, not task-specific language data. We then place this model in a multi-agent self-play environment that gen- erates task-specific rewards used to adapt or modulate the model, turning it into a task- conditional language model. We introduce a new way for combining the two types of learn- ing based on the idea of reranking language model samples, and show that this method outperforms others in communicating with hu- mans in a visual referential communication task. Finally, we present a taxonomy of dif- ferent types of language drift that can occur alongside a set of measures to detect them. 1 Introduction In this work, we aim at making agents communi- cate with humans in natural language. Our starting point is a language model that has been trained on generic, not task-specific language data. We then place this model in a multi-agent communication environment that generates task-specific rewards, which are used to adapt or modulate the model, making it task-conditional. We thus propose to de- compose the problem of learning language use into two components: learning “what” to say based on a given situation, and learning “how” to say it. The “what” is the essence of communication that under- lies our intentions and is chosen by maximizing any given utility, making it a functional, utility-driven process. On the other hand, the “how” is a surface realization of our intentions, i.e., the words we use ∗All authors contributed equally. to communicate this “what” successfully. This fac- torization into content planning (here, “what”) and surface realization (here, “how”) moves us away from end-to-end neural generation systems and is in line with traditional methods of natural language generation (Reiter and Dale, 1997). More impor- tantly, it enables us to bring together two different strands of research: traditional data-driven natural language learning and multi-agent communication. Traditional approaches to natural language learn- ing (Kneser and Ney, 1995; Mikolov et al., 2010; Sutskever et al., 2014; Vinyals and Le, 2015; Rad- ford et al., 2019) are based on inferring structural properties of language from text corpora, often in a passive regime, dissociated from communication. While this type of learning is great for learning gen- eral statistical associations between symbols (e.g., adjectives come before nouns) and even inferring semantic relations, it ignores the functional aspects of communication, i.e., the fact that people use words to coordinate with others and make things happen in the world (Wittgenstein, 1953; Austin, 1975; Clark, 1996). On the other hand, multi-agent …

Content truncated to 500 words