UMUTeam at SemEval-2022 Task 6: Evaluating Transformers for detecting Sarcasm in English and Arabic

Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 1012 - 1017 July 14-15, 2022 ©2022 Association for Computational Linguistics UMUTeam at SemEval-2022 Task 6: Evaluating Transformers for detecting Sarcasm in English and Arabic José Antonio García-Díaz and Camilo Caparrós-Laiz and Rafael Valencia-García∗ Facultad de Informática, Universidad de Murcia, Campus de Espinardo, 30100, Spain {joseantonio.garcia8,camilo.caparrosl,valencia}@um.es Abstract In this manuscript we detail the participation of the UMUTeam in the iSarcasm shared task (SemEval-2022). This shared task is related to the identification of sarcasm in English and Arabic documents. Our team achieve in the first challenge, a binary classification task, a F1 score of the sarcastic class of 17.97 for English and 31.75 for Arabic. For the second challenge, a multi-label classification, our results are not recorded due to an unknown problem. There- fore, we report the results of each sarcastic mechanism with the validation split. For our proposal, several neural networks that combine language-independent linguistic features with pre-trained embeddings are trained. The em- beddings are based on different schemes, such as word and sentence embeddings, and contex- tual and non-contextual embeddings. Besides, we evaluate different techniques for the inte- gration of the feature sets, such as ensemble learning and knowledge integration. In general, our best results are achieved using the knowl- edge integration strategy. 1 Introduction Sarcasm is a form of rhetorical device based on biting humour to disarm an opponent during a di- alog (Wilson, 2006). On the Web and, specially in social networks and opinion forums, sarcasm is very popular because it is funny to read and helps to stimulate the viral phenomenon of social media content (Peng et al., 2019). As sarcasm usually re- lies on figurative language and wordplay, in which words diverts from their conventional meaning, sar- castic statements hinder the ability of automatic classification tasks to perform sentiment analysis, hate-speech detection, or author analysis among other tasks. From a Natural Language Processing (NLP) per- spective, sarcasm, among other forms of figura- tive speech, such as irony or satire, has been ex- ∗Corresponding author plored in (del Pilar Salas-Zárate et al., 2020). In this work, the authors explored what stands out the most discriminant features for satire and irony de- tection. The authors identified a total of 25 feature sets. Apart from lexicon-based features and word- n-grams features, such as unigrams or bigrams, the authors identified style features, sentiment and emotional features, pragmatic features, and punc- tuation features, to name a few. Our proposal for solving sarcasm detection includes language- independent feature sets extracted with a custom tool, UMUTextStats (García-Díaz et al., 2021; García-Díaz and Valencia-García, 2022; García- Díaz et al., 2022a,b). In this work, we describe the participation of the UMUTeam at SemEval 2022 task 6, concern- ing sarcasm identification in Arabic and English (Abu Farha et al., 2022). In this edition, three chal- lenges are proposed. The first one is a binary classi- fication problem to determine whether a document is sarcastic or not. This challenge is for English and Arabic. The …

Content truncated to 500 words