What Meaning-Form Correlation Has to Compose With: A Study of MFC on Artificial and Natural Language

Proceedings of the 28th International Conference on Computational Linguistics, pages 3737–3749 Barcelona, Spain (Online), December 8-13, 2020 3737 What Meaning-Form Correlation Has to Compose With: A Study of MFC on Artificial and Natural Language Timothee Mickus∗ ATILF tmickus@atilf.fr Timoth´ee Bernard∗ Universit´e de Paris timothee.bernard @ens-lyon.org Denis Paperno Utrecht University d.paperno@uu.nl Abstract Compositionality is a widely discussed property of natural languages, although its exact def- inition has been elusive. We focus on the proposal that compositionality can be assessed by measuring meaning-form correlation. We analyze meaning-form correlation on three sets of lan- guages: (i) artificial toy languages tailored to be compositional, (ii) a set of English dictionary definitions, and (iii) a set of English sentences drawn from literature. We find that linguistic phe- nomena such as synonymy and ungrounded stop-words weigh on MFC measurements, and that straightforward methods to mitigate their effects have widely varying results depending on the dataset they are applied to. Data and code are made publicly available. 1 Introduction Compositionality is one of the core aspects of language: when speaking, we weave complex propositions out of unanalyzable atoms of meaning. This intricate interplay between symbols and sentences has been an important point of research in areas ranging from philosophy of language (Frege, 1884) to distribu- tional semantics (Baroni et al., 2014), and has been held as one of the key characteristic differentiating human language from animal communication (Hockett, 1960). Compositionality in natural language is usually captured by the Principle of Compositionality. This principle, often attributed to Frege although this paternity is debatable (Janssen, 2012), has been worded in different ways; let us quote a classic text- book (Gamut, 1991, p.26): “the meaning of a composite expression must be wholly determined by the meanings of its composite parts and of the syntactic rule by means of which it is formed”. One suggestion, that can be traced back to Kirby (1999), and that was fully operationalized in Kirby et al. (2008), is that compositionality can be measured as a correlation between meaning and surface form (i.e., the sequence of tokens). If we consider sentences such as “I saw a black cat” and “I saw a black crow”, we see that minute changes in the form of a sentence entail that its meaning does not change too much. Compare these examples to sentences such as “I saw a white crow”, or “He saw a white crow”: the gradual alteration of the form of a sentence also gradually alters its meaning. Provided that we can measure similarity between any two sentences forms, and between any two sentences meanings, this observation can be rephrased as follows: the distance between two sentences meanings should correlate with the distance between their forms. We refer to this as meaning-form correlation, or MFC for short. One area where MFC has been employed as a way to both detect and quantify compositionality is the field of emergent communication (Kirby et al., 2008; Kirby et al., 2015; Spike, 2016; Ren et al., 2020, a.o.) which studies agents (artificial …

Content truncated to 500 words