The Grammar of Emergent Languages

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 3339–3359, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 3339 The Grammar of Emergent Languages Oskar van der Wal University of Amsterdam oskar.vanderwal@gmail.com Silvan de Boer University of Amsterdam silvandeboer@gmail.com Elia Bruni∗ Institute of Cognitive Science University of Osnabruek elia.bruni@gmail.com Dieuwke Hupkes∗ Institute for Logic, Language and Computation University of Amsterdam d.hupkes@uva.nl Abstract In this paper, we consider the syntactic proper- ties of languages emerged in referential games, using unsupervised grammar induction (UGI) techniques originally designed to analyse nat- ural language. We show that the considered UGI techniques are appropriate to analyse emergent languages and we then study if the languages that emerge in a typical referential game setup exhibit syntactic structure, and to what extent this depends on the maximum mes- sage length and number of symbols that the agents are allowed to use. Our experiments demonstrate that a certain message length and vocabulary size are required for structure to emerge, but they also illustrate that more so- phisticated game scenarios are required to ob- tain syntactic properties more akin to those observed in human language. We argue that UGI techniques should be part of the standard toolkit for analysing emergent languages and release a comprehensive library to facilitate such analysis for future researchers. 1 Introduction Artificial agents parameterised by deep neural net- works can learn to communicate using discrete symbols to solve collaborative tasks (Foerster et al., 2016; Lazaridou et al., 2017; Havrylov and Titov, 2017). A prime reason to conduct such studies, which constitute a new generation of experiments with referential games, is that they may provide insight in the factors that shaped the evolution of human languages (Kirby, 2002). However, the emergent languages developed by neural agents are not human-interpretable, and lit- tle is known about their semantic and syntactic ∗Shared senior authorship nature. More specifically, we do not know to what extent the structure of emergent languages resem- bles the structure of human languages, what the lan- guages encode, and how these two things depend on choices that need to be made by the modeller. A substantial obstacle to better understanding emergent languages is the lack of tools to analyse their properties. Previous work has concentrated primarily on understanding languages through their semantics, by studying the alignment of messages and symbolic representations of the meaning space (e.g. Lazaridou et al., 2018). A substantial down- side of such approaches is that they are restricted to scenarios for which a symbolic representation of the meaning space is available. Furthermore, they ignore a second important aspect of language: syntax, which is relevant not just for syntactically- oriented researchers, but also for those that are interested in semantics from a compositional per- spective. In this work, we aim to address this gap in the literature by presenting an analysis of the syntax of emergent languages. We take inspiration from unsupervised grammar induction (UGI) techniques originally proposed for natural language. In particular, we use them to investigate …

Content truncated to 500 words