R2D2 at SemEval-2022 Task 6: Are language models sarcastic enough? Finetuning pre-trained language models to identify sarcasm

Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 1018 - 1024 July 14-15, 2022 ©2022 Association for Computational Linguistics R2D2 at SemEval-2022 Task 6: Are language models sarcastic enough? Finetuning pre-trained language models to identify sarcasm Mayukh Sharma, Ilanthenral Kandasamy and W.B. Vasantha School of Computer Science and Engineering Vellore Institute of Technology Vellore - 632014, Tamil Nadu, India 04mayukh@gmail.com,ilanthenral.k@vit.ac.in, vasantha.wb@vit.ac.in Abstract This paper describes our system used for Se- mEval 2022 Task 6: iSarcasmEval: Intended Sarcasm Detection in English and Arabic. We participated in all subtasks based on only En- glish datasets. Pre-trained Language Models (PLMs) have become a de-facto approach for most natural language processing tasks. In our work, we evaluate the performance of these models for identifying sarcasm. For Subtask A and Subtask B, we used simple finetun- ing on PLMs. For Subtask C, we propose a Siamese network architecture trained using a combination of cross-entropy and distance- maximisation loss. Our model was ranked 7th in Subtask B, 8th in Subtask C (English), and performed well in Subtask A (English). In our work, we also present the comparative perfor- mance of different PLMs for each Subtask. 1 Introduction Sarcasm, a form of figurative speech, allows people to express contempt or mock using irony. The irony is used to communicate the opposite of the intended meaning to express humour or mock something. Sarcasm plays an essential role in people’s daily conversation and finds its use across social media to express thoughts. Recently, social media has drawn in millions of users around the world. Owing to its figurative nature, sarcasm poses a significant challenge to systems performing sentiment anal- ysis on these social media platforms. Therefore, it is essential to design and develop systems that efficiently identify sarcasm. The most challeng- ing aspect of sarcasm is the different socio-cultural backgrounds of people who drive it. Therefore, sarcasm intended by the author may not always be perceived by the audience from different back- grounds. Most datasets on sarcasm detection are collected using predefined criteria or human anno- tators (Oprea and Magdy, 2020). This is a sub- optimal approach that may not always capture the sarcasm intended by the author leading to noise in the models trained on it. SemEval 2022 Task 6: iSarcasmEval: Intended Sarcasm Detection in English and Arabic (Abu Farha et al., 2022) draws attention to the problem of identifying sarcasm us- ing a dataset of intended sarcasm. Our proposed approach for Subtask A and Sub- task B approach uses a classification objective to finetune PLMs. PLMs learn semantic and syntac- tic features via training on large amounts of a text corpus. This information is used in downstream tasks by simply finetuning task-specific datasets. PLMs have shown remarkable performance on such downstream tasks using the simple finetun- ing approach (Sharma et al., 2021b). We extend the same idea to identify sarcasm (Subtask A) and iden- tify the type of irony (Subtask B). Subtask C aimed at identifying sarcastic text from its non-sarcastic counterpart. We propose …

Content truncated to 500 words