LT3 at SemEval-2022 Task 6: Fuzzy-Rough Nearest Neighbor Classification for Sarcasm Detection

Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 1 - 14 July 14-15, 2022 ©2022 Association for Computational Linguistics Semeval-2022 Task 1: CODWOE – Comparing Dictionaries and Word Embeddings Timothee Mickus∗ Helsinki University timothee.mickus @helsinki.fi Kees van Deemter Utrecht University c.j.vandeemter @uu.nl Mathieu Constant Université de Lorraine CNRS, ATILF mconstant @atilf.fr Denis Paperno Utrecht University d.paperno @uu.nl Abstract Word embeddings have advanced the state of the art in NLP across numerous tasks. Under- standing the contents of dense neural represen- tations is of utmost interest to the computa- tional semantics community. We propose to focus on relating these opaque word vectors with human-readable definitions, as found in dictionaries. This problem naturally divides into two subtasks: converting definitions into embeddings, and converting embeddings into definitions. This task was conducted in a mul- tilingual setting, using comparable sets of em- beddings trained homogeneously. 1 Introduction Word embeddings are a success story in NLP. They have been equated to distributional semantics mod- els (Lenci, 2018; Boleda, 2020), a theory of seman- tics which relates the meaning of words to their distribution in context (Harris, 1954). Recently introduced contextualized word embeddings (e.g. Devlin et al., 2019) have set a new state of the art on a wide variety of tasks. For this reason, they have attracted much research interest. Do they depict consistent semantic spaces and are they theoreti- cally valid (Mickus et al., 2020b; Yenicelik et al., 2020)? What limitations are to be expected in these models (Bender and Koller, 2020)? Can they scale up in performance (Brown et al., 2020)? Word embeddings are dense vector representa- tions of meaning which are not easily intelligible to a human observer. Many techniques have been employed to make embedding spaces more inter- pretable. A promising approach consists in con- verting these opaque vectors into human readable definitions, as one could find in a dictionary: ac- curately translating a dense, opaque vector repre- sentation into an equivalent human-readable piece of text would allow us to peer into the black box ∗Work conducted while at ATILF Figure 1: Logo for CODWOE shared task of modern neural network architectures. This av- enue of research, known as definition modeling, was pioneered by Noraset et al. (2017). One may however question whether the task is at all feasible: there is no guarantee that the information content of a dictionary definition is similar to that which is described by real-valued vectors inferred from word distributions. The SemEval Shared Task on Comparing Dictio- naries and Word Embeddings (CODWOE) sets out to study whether embeddings and dictionaries en- code similar information. We present the task and relevant state of the art in Section 2. We describe the data collected and presented to participants in Section 3. In Section 4, we discuss the metrics used to rank participant submissions. Our baseline model is presented in Section 5. We list results from participants’ submissions in Section 6 and provide a more in-depth discussion in Section 7. 2 What we are fishing for …

Content truncated to 500 words