Assessing the Representations of Idiomaticity in Vector Models with a Noun Compound Dataset Labeled at Type and Token Levels

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 2730–2741 August 1–6, 2021. ©2021 Association for Computational Linguistics 2730 Assessing the Representations of Idiomaticity in Vector Models with a Noun Compound Dataset Labeled at Type and Token Levels Marcos Garcia∗1, Tiago Kramer Vieira∗2, Carolina Scarton3, Marco A. P. Idiart4, Aline Villavicencio2,3 1 CiTIUS Research Centre, Universidade de Santiago de Compostela, Galiza (Spain) 2 Institute of Informatics, Federal University of Rio Grande do Sul (Brazil) 3 Department of Computer Science, University of Sheffield (UK) 4 Institute of Physics, Federal University of Rio Grande do Sul (Brazil) marcos.garcia.gonzalez@udc.gal, tiagokv@hotmail.com, {c.scarton, a.villavicencio}@sheffield.ac.uk, marco.idiart@gmail.com Abstract Accurate assessment of the ability of embed- ding models to capture idiomaticity may re- quire evaluation at token rather than type level, to account for degrees of idiomaticity and pos- sible ambiguity between literal and idiomatic usages. However, most existing resources with annotation of idiomaticity include ratings only at type level. This paper presents the Noun Compound Type and Token Idiomatic- ity (NCTTI) dataset, with human annotations for 280 noun compounds in English and 180 in Portuguese at both type and token level. We compiled 8,725 and 5,091 token level an- notations for English and Portuguese, respec- tively, which are strongly correlated with the corresponding scores obtained at type level. The NCTTI dataset is used to explore how vector space models reflect the variability of idiomaticity across sentences. Several ex- periments using state-of-the-art contextualised models suggest that their representations are not capturing the noun compounds idiomatic- ity as human annotators. This new multilin- gual resource also contains suggestions for paraphrases of the noun compounds both at type and token levels, with uses for lexical sub- stitution or disambiguation in context. 1 Introduction Multiword Expressions (MWEs) such as noun compounds (NCs), have been considered a chal- lenge for NLP (Sag et al., 2002). This is partly due to the wide range of idiomaticity that they display, from more literal to idiomatic combinations (olive oil vs. shrinking violet). The task of identifying the degree of idiomaticity of MWEs has been investi- gated at type level, to determine the potential of an MWE to be idiomatic in general. Some of these approaches are based on the assumption that the * Equal contribution. distance between the representation of an MWE as a unit and the representation of the compositional combination of its components is an indication of the degree of idiomaticity: they are closer if the MWE is more compositional. Good performances are obtained even with non-contextualised word embeddings like word2vec (Mikolov et al., 2013), and vector operations like addition and multipli- cation (Mitchell and Lapata, 2010; Reddy et al., 2011; Cordeiro et al., 2019). Additionally, for some MWEs, there is a potential ambiguity be- tween an idiomatic and a literal sense, like in the potentially idiomatic MWE brass ring which can be ambiguous between the more literal meaning a ring made of brass and the more idiomatic sense …

Content truncated to 500 words