Compositionality and Generalization In Emergent Languages

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4427–4442 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 4427 Compositionality and Generalization in Emergent Languages Rahma Chaabouni1,2∗, Eugene Kharitonov1∗, Diane Bouchacourt1, Emmanuel Dupoux1,2, and Marco Baroni1,3 1Facebook AI Research 2Cognitive Machine Learning (ENS - EHESS - PSL Research University - CNRS - INRIA) 3ICREA {rchaabouni,kharitonov,dianeb,dpx,mbaroni}@fb.com Abstract Natural language allows us to refer to novel composite concepts by combining expressions denoting their parts according to systematic rules, a property known as compositional- ity. In this paper, we study whether the lan- guage emerging in deep multi-agent simula- tions possesses a similar ability to refer to novel primitive combinations, and whether it accomplishes this feat by strategies akin to human-language compositionality. Equipped with new ways to measure compositionality in emergent languages inspired by disentan- glement in representation learning, we estab- lish three main results. First, given sufficiently large input spaces, the emergent language will naturally develop the ability to refer to novel composite concepts. Second, there is no cor- relation between the degree of compositional- ity of an emergent language and its ability to generalize. Third, while compositionality is not necessary for generalization, it provides an advantage in terms of language transmis- sion: The more compositional a language is, the more easily it will be picked up by new learners, even when the latter differ in archi- tecture from the original agents. We conclude that compositionality does not arise from sim- ple generalization pressure, but if an emergent language does chance upon it, it will be more likely to survive and thrive. 1 Introduction Most concepts we need to express are composite in some way. Language gives us the prodigious ability to assemble messages referring to novel composite concepts by systematically combining expressions denoting their parts. As interest raises in develop- ing deep neural agents evolving a communication code to better accomplish cooperative tasks, the question arises of how the emergent code can be ∗Contributed equally. endowed with the same desirable compositionality property (Kottur et al., 2017; Lazaridou et al., 2018; Mordatch and Abbeel, 2018; Cogswell et al., 2019; Li and Bowling, 2019). This in turn requires mea- sures of how compositional an emergent language is (Andreas, 2019). Compositionality is a core no- tion in linguistics (Partee, 2004), but linguists’ def- initions assume full knowledge of primitive expres- sions and their combination rules, which we lack when analyzing emergent languages (Nefdt, 2020). Also, these definitions are categorical, whereas to compare emergent languages we need to quantify degrees of compositionality. Some researchers equate compositionality with the ability to correctly refer to unseen composite inputs (e.g., Kottur et al., 2017; Cogswell et al., 2019). This approach measures the generalization ability of a language, but it does not provide any insights on how this ability comes about. Indeed, one of our main results below is that emergent languages can attain perfect generalization without abiding to intuitive notions of compositionality. Topographic similarity has become the standard way to quantify the …

Content truncated to 500 words