UTNLP at SemEval-2022 Task 6: A Comparative Analysis of Sarcasm Detection Using Generative-based and Mutation-based Data Augmentation

Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 962 - 969 July 14-15, 2022 ©2022 Association for Computational Linguistics UTNLP at SemEval-2022 Task 6: A Comparative Analysis of Sarcasm Detection Using Generative-based and Mutation-based Data Augmentation Amirhossein Abaskohi, Arash Rasouli∗, Tanin Zeraati∗, Behnam Bahrak School of Electrical and Computer Engineering, College of Engineering University of Tehran, Tehran, Iran {amir.abaskohi, arash.rasouli, t.zeraati, bahrak}@ut.ac.ir Abstract Sarcasm is a term that refers to the use of words to mock, irritate, or amuse someone. It is com- monly used on social media. The metaphorical and creative nature of sarcasm presents a signif- icant difficulty for sentiment analysis systems based on affective computing. The methodol- ogy and results of our team, UTNLP, in the SemEval-2022 shared task 6 on sarcasm detec- tion are presented in this paper. We put differ- ent models, and data augmentation approaches to the test and report on which one works best. The tests begin with traditional machine learn- ing models and progress to transformer-based and attention-based models. We employed data augmentation based on data mutation and data generation. Using RoBERTa and mutation- based data augmentation, our best approach achieved an F1-sarcastic of 0.38 in the competi- tion’s evaluation phase. After the competition, we fixed our model’s flaws and achieved an F1-sarcastic of 0.414. 1 Introduction Billions of internet users use social networks not only to stay in touch with friends, meet new peo- ple, and share user-generated content but also to express their opinions on a wide range of topics using a variety of methods such as posting com- ments, videos, photos, etc. with specific groups of people (Tungthamthiti et al., 2016). In these plat- forms, users could submit information on whatever topic they wanted, with no restrictions on the sort of content they may share. The lack of constraints and individuals’ anonymity on these networks led to humorous sarcastic texts. Because sarcasm indicates sentiment, detecting sarcasm in a text is critical for anticipating the text’s accurate sentiment, making sarcasm detec- tion a valuable tool with multiple applications in domains such as security, health, services, prod- uct evaluations, and sales. Sarcasm detection is an ∗equal contribution essential aspect of creative language comprehen- sion (Veale et al., 2019) and online opinion mining (Kannangara, 2018). Even for humans, identifying sarcasm is difficult due to heavily contextualized expressions (Walker et al., 2012). There are few labeled data resources for sarcasm detection. Any available texts that can be collected (for example, Tweets) contain many issues, such as an evolving dictionary of slang words and abbreviations, requir- ing many hours of human annotation to prepare the data for any potential use. Furthermore, the nature of sarcasm identification adds to the task’s difficulty, as sarcasm may be considered relative and varies significantly across people, depending on a variety of criteria such as the context, area, time, and events surrounding the statement. In an attempt to solve this issue, we participated in SemEval-2022 shared task 6 (Abu Farha et al., 2022), which aims to …

Content truncated to 500 words