reamtchka at SemEval-2022 Task 6: Investigating the effect of different loss functions for Sarcasm detection for unbalanced datasets

Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 896 - 906 July 14-15, 2022 ©2022 Association for Computational Linguistics reamtchka at SemEval-2022 Task 6: Investigating the effect of different loss functions for Sarcasm detection for unbalanced datasets Reem Abdel-Salam Computer Engineering Department Cairo University reem.abdelsalam13@gmail.com Abstract This paper describes the system used in SemEval-2022 Task 6: Intended Sarcasm De- tection in English and Arabic. Achieving 20th, 3rd places with 34& 47 F1-Sarcastic score for task A, 16th place for task B with 0.0560 F1- macro score, and 10, 6th places for task C with 72% and 80% accuracy on the leaderboard. A voting classifier between either multiple differ- ent BERT-based models or machine learning models is proposed, as our final model. Multi- ple key points have been extensively examined to overcome the problem of the unbalance of the dataset as: type of models, suitable archi- tecture, augmentation, loss function, etc. In addition to that, we present an analysis of our results in this work, highlighting its strengths and shortcomings. 1 Introduction Sarcasm detection in any language text is hard for many reasons. First Sarcasm is much more than just written words. It is the tone, emphasis, experi- ence, personal knowledge, and even facial expres- sions and body language that convey the meaning. When written into text such information is lost com- pletely. Another reason is cultural references which make it hard for non-natives to get. In addition, all that is needed to show sarcasm is to be clear and articulate and not be as sketchy and without for- matting, punctuation, proper use of language as texting might be for some people in their usual use of such media. Sarcasm is widespread on the social web and, by definition, may be extremely disrup- tive to machine learning/ deep learning models that use this data to perform tasks such as sentiment analysis, opinion mining, author profiling, and ha- rassment identification. Thus, sarcasm detection might be the first crucial step in these systems. Sev- eral methods have been proposed to detect sarcasm in text as the recent shared-task for sarcasm detec- tion in Arabic language (Abu Farha et al., 2021), where several teams used pretrained language mod- els such as AraBERT and MarBert (Wadhawan, 2021; Song et al., 2021; Naski et al., 2021; Faraj et al., 2021; Abdel-Salam, 2021; Abuzayed and Al-Khalifa, 2021). In some of the work proposed in English language sarcasm detection involved rule-based and statistical approaches using: (a) Un- igrams and pragmatic features (for example emoti- cons, etc.) (b) Sentiment and polarity estimation (c) Extraction of common patterns (Nagwanshi and Madhavan, 2014; Kumar et al., 2020; Bouazizi and Ohtsuki, 2016). A Recent shared-task for En- glish language (Ghosh et al., 2020), where several methods has been proposed were based on pre- trained model (i.e. BERT and RoBERTa) (Baruah et al., 2020; A. and D., 2020; Amir et al., 2016; Shangipour ataei et al., 2020). In SemEval-2022 Task 6 (Abu Farha et al., 2022), the goal is to …

Content truncated to 500 words