When did you become so smart, oh wise one?! Sarcasm Explanation in Multi-modal Multi-party Dialogues

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 5956 - 5968 May 22-27, 2022 c⃝2022 Association for Computational Linguistics When did you become so smart, oh wise one?! Sarcasm Explanation in Multi-modal Multi-party Dialogues Shivani Kumar∗, Atharva Kulkarni∗, Md Shad Akhtar, Tanmoy Chakraborty Indraprastha Institute of Information Technology Delhi, India {shivaniku, atharvak, shad.akhtar, tanmoy}@iiitd.ac.in Abstract Indirect speech such as sarcasm achieves a constellation of discourse goals in human com- munication. While the indirectness of figu- rative language warrants speakers to achieve certain pragmatic goals, it is challenging for AI agents to comprehend such idiosyncrasies of human communication. Though sarcasm identification has been a well-explored topic in dialogue analysis, for conversational systems to truly grasp a conversation’s innate mean- ing and generate appropriate responses, sim- ply detecting sarcasm is not enough; it is vi- tal to explain its underlying sarcastic connota- tion to capture its true essence. In this work, we study the discourse structure of sarcastic conversations and propose a novel task – Sar- casm Explanation in Dialogue (SED). Set in a multimodal and code-mixed setting, the task aims to generate natural language explanations of satirical conversations. To this end, we cu- rate WITS, a new dataset to support our task. We propose MAF (Modality Aware Fusion), a multimodal context-aware attention and global information fusion module to capture multi- modality and use it to benchmark WITS. The proposed attention module surpasses the tra- ditional multimodal fusion baselines and re- ports the best performance on almost all met- rics. Lastly, we carry out detailed analyses both quantitatively and qualitatively. 1 Introduction The use of figurative language serves many com- municative purposes and is a regular feature of both oral and written communication (Roberts and Kreuz, 1994). Predominantly used to induce hu- mour, criticism, or mockery (Colston, 1997), para- doxical language is also used in concurrence with hyperbole to show surprise (Colston and Keller, 1998) as well as highlight the disparity between ex- pectations and reality (Ivanko and Pexman, 2003). While the use and comprehension of sarcasm is a ∗Equal contribution Figure 1: Sarcasm Explanation in Dialogues (SED). Given a sarcastic dialogue, the aim is to generate a nat- ural language explanation for the sarcasm in it. Blue text represents the English translation for the text. cognitively taxing process (Olkoniemi et al., 2016), psychological evidence advocate that it positively correlates with the receiver’s theory of mind (ToM) (Wellman, 2014), i.e., the capability to interpret and understand another person’s state of mind. Thus, for NLP systems to emulate such anthropomorphic intelligent behavior, they must not only be potent enough to identify sarcasm but also possess the ability to comprehend it in its entirety. To this end, moving forward from sarcasm identification, we propose the novel task of Sarcasm Explanation in Dialogue (SED). For dialogue agents, understanding sarcasm is even more crucial as there is a need to normalize its sarcastic undertone and deliver appropriate re- sponses. Conversations interspersed with sarcastic statements often use contrastive …

Content truncated to 500 words