MarSan at SemEval-2022 Task 6: iSarcasm Detection via T5 and Sequence Learners

Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 978 - 986 July 14-15, 2022 ©2022 Association for Computational Linguistics MarSan at SemEval-2022 Task 6: iSarcasm Detection via T5 and Sequence Learners Maryam Najafi1,∗, Ehsan Tavan1,∗ 1 NLP Department, Part AI Research Center, Tehran, Iran {maryam.najafi, ehsan.tavan}@partdp.ai Abstract The paper describes SemEval-2022’s shared task "Intended Sarcasm Detection in English and Arabic." This task includes English and Arabic tweets with sarcasm and non-sarcasm samples and irony speech labels. The first two subtasks predict whether a text is sarcas- tic and the ironic category the sarcasm sam- ple belongs to. The third one is to find the sarcastic sample from a sarcastic sample and its non-sarcastic paraphrase. Deep neural net- works have recently achieved highly compet- itive performance in many tasks. Combining deep learning with language models has also re- sulted in acceptable accuracy. Inspired by this, we propose a novel deep learning model on top of language models. On top of T5, the architec- ture uses an encoder module of the transformer, followed by LSTM and attention to utilizing past and future information, concentrating on informative tokens. Due to the success of the proposed model, we used the same architecture with a few modifications to the output layer in all three subtasks. 1 Introduction Sarcasm is a sophisticated form of expression that implicitly conveys the content of a sentence. Au- tomated sarcasm detection focuses mainly on the lexical, syntactic, and semantic levels of text analy- sis Hazarika et al. (2018). Natural language understanding(NLU), dialogue systems, and text mining can benefit from sarcasm detection. Arguably, the most challenging part of sarcasm is its rarity, infrequency, difficulty in de- tecting, and ambiguity in meaning. Sarcasm, for instance, can imply a negative meaning with the use of positive words. For example, "Taxes are just the best, and I cannot wait to pay more " a sarcastic sentence that uses positive words but carries a negative meaning of "I dislike pay- ing taxes." As a result, detecting sarcasm poses a ∗Equal contribution. Listing order is random. challenging task due to the nature of sarcastic texts, which are influenced by several factors, such as context, region, and mentality. A sarcasm detection algorithm goes beyond sen- timent analysis, and instead of looking at sentiment in a sample, it focuses on sarcasm. The purpose of this field is to identify whether a given text is sarcastic or not. Recently, it has been shown that neural language models trained on unstructured text can implic- itly store and retrieve knowledge. Studies have re- vealed that the Text-To-Text Transfer Transformer (T5) architecture Raffel et al. (2019) can achieve high performance for various NLP applications. An essential step in creating NLP models is choos- ing an appropriate embedding vector. In this re- search, the T5 and Multilingual T5 (MT5) (Xue et al., 2020) Encoder module for English and Ara- bic was implemented, respectively. Our task is comprised of three Subtasks: • Subtask A: Predict the sarcastic nature (sar- castic or …

Content truncated to 500 words