Vectorial Semantic Spaces Do Not Encode Human Judgments of Intervention Similarity

Proceedings of the 22nd Conference on Computational Natural Language Learning (CoNLL 2018), pages 392–401 Brussels, Belgium, October 31 - November 1, 2018. c⃝2018 Association for Computational Linguistics 392 Vectorial semantic spaces do not encode human judgments of intervention similarity Paola Merlo and Francesco Ackermann Department of Linguistics University of Geneva 5 Rue de Candolle, CH-1211 Gen`eve 4 paola.merlo@unige.ch, francesco.ackermann@unige.ch Abstract Despite their practical success and impres- sive performances, neural-network-based and distributed semantics techniques have often been criticized as they remain fundamentally opaque and difficult to interpret. In a vein sim- ilar to recent pieces of work investigating the linguistic abilities of these representations, we study another core, defining property of lan- guage: the property of long-distance depen- dencies. Human languages exhibit the abil- ity to interpret discontinuous elements distant from each other in the string as if they were adjacent. This ability is blocked if a similar, but extraneous, element intervenes between the discontinuous components. We present re- sults that show, under exhaustive and precise conditions, that one kind of word embeddings and the similarity spaces they define do not en- code the properties of intervention similarity in long-distance dependencies, and that there- fore they fail to represent this core linguistic notion. 1 Introduction Despite their practical success and impressive per- formances, neural-network-based and distributed semantics techniques have often been criticized as they remain fundamentally opaque and difficult to interpret. To cast light on what linguistic information is learnt and encoded in these representations, sev- eral pieces of work have recently studied core properties of language in syntax (Linzen et al., 2016; Bernardy and Lappin, 2017; Gulordava et al., 2018; Linzen and Leonard, 2018; van Schijndel and Linzen, 2018), semantics (Herbelot and Ganesalingam, 2013; Erk, 2016), morphology (Cotterell and Sch¨utze, 2015). In a similar vein, we study another core, defining property of human languages: the property of long-distance depen- dencies. Human languages exhibit the ability to interpret discontinuous elements distant from each other in the string as if they were adjacent.1 Sentence (1a) is a question about the object of the verb buy, whose canonical position is shown in angle brack- ets, thus connecting the first and last element in the sentence.2 Sentence (2a) is a relative clause where the object of the verb wash is also the semantic object of the verb show, connecting two distant elements. Sentence (3a) is also a relative clause where the word ´etudiant (student) is the semantic object of the verb endort (put to sleep). (1a) What do you wonder John bought <what> ? (2a) Show me the elephant that the lion is wash- ing <the elephant>. (3a) Jules sourit aux ´etudiants que l’orateur endort < ´etudiants> s´erieusement depuis le d´ebut. ’Jules smiles to the students who the speaker is putting seriously to sleep from the begin- ning.’ Long-distance dependencies are not all equally acceptable. The precise description of the facts involving long-distance dependencies is complex, and is one of the major topics of research in current linguistic theory, with many competing proposals 1To …

Content truncated to 500 words