TechSSN at SemEval-2022 Task 6: Intended Sarcasm Detection using Transformer Models

Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 94 - 100 July 14-15, 2022 ©2022 Association for Computational Linguistics Abstract This paper describes our two deep learning systems that competed at SemEval-2022 Task 1 “CODWOE: Comparing Dictionaries and WOrd Embeddings”. We participated in the subtask for the reverse dictionary which consists in generating vectors from glosses. We use sequential models that integrate several neural networks, starting from Embeddings networks until the use of Dense networks, Bidirectional Long Short-Term Memory (BiLSTM) networks and LSTM networks. All glosses have been preprocessed in order to consider the best representation form of the meanings for all words that appears. We achieved very competitive results in reverse dictionary with a second position in English and French languages when using contextualized embeddings, and the same position for English, French and Spanish languages when using char embeddings. Our source code can be found at GitHub1. 1 Introduction Distributed representations of words (or word embeddings) (Bengio et al., 2003; Mikolov et al., 2013; Pennington, Socher and Manning, 2014) have shown to provide useful features for various tasks in natural language processing (NLP) and computer vision. While there seems to be a consensus concerning the usefulness of word embeddings and how to learn them, this is not yet clear with regard to representations that carry the meaning of a full sentence. That is, how to capture 1 https://github.com/jln-brtn/BL.Research-at- SemEval-2022-Task-1 the relationships among multiple words and phrases in a single vector remains a question to be solved. Much recent research in computational semantics has focused on learning representations of arbitrary-length phrases and sentences. The reverse dictionary represents one of the most common cases to solve this problem of learning sequence representations. That said, the reverse dictionary is the task to find the proper target word given the word description (Hill et al., 2016; Hedderich et al., 2019; Zhang et al., 2019; Yan, Li and Qiu, 2020). For example, the composed meaning of the words in a dictionary definition (A mixture of other substances or things) should correspond to the meaning of the word that define it (cocktail). As mentioned by Hill et al. (2016), this bridge between lexical and phrasal semantics is useful because high quality vector representations of single words can be used as a target when learning to combine the words into a coherent phrasal representation. In this paper, we present our contributions to solve the reverse dictionary problem using very specific neural architectures and applying supervised learning. For more information on task 1 of SemEval-2022 as described by its organizers, we invite the reader to consult the paper of Mickus, Timothee et al. (2022). Our approaches require a model able of learning to map between arbitrary-length phrases and fixed- length continuous-valued word vectors. For this purpose, we experiment with two broad classes of neural language models (NLMs): Recurrent Neural Networks (RNNs) with (Bidirectional) Long-Short BL.Research at SemEval-2022 Task 1: Deep networks for Reverse Dictionary using embeddings and LSTM autoencoders Nihed Bendahman†, Julien Breton†, Lina …

Content truncated to 500 words