All-in-One: A Deep Attentive Multi-task Learning Framework for Humour, Sarcasm, Offensive, Motivation, and Sentiment on Memes

Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing, pages 281–290 December 4 - 7, 2020. c⃝2020 Association for Computational Linguistics 281 All-in-One: A Deep Attentive Multi-task Learning Framework for Humour, Sarcasm, Offensive, Motivation, and Sentiment on Memes Dushyant Singh Chauhan, Dhanush S R, Asif Ekbal and Pushpak Bhattacharyya Department of Computer Science & Engineering Indian Institute of Technology Patna Patna, Bihar, India-801106 {1821CS17, dhanush.cs16, asif, pb}@iitp.ac.in Abstract In this paper, we aim at learning the relation- ships and similarities of a variety of tasks, such as humour detection, sarcasm detection, offen- sive content detection, motivational content de- tection and sentiment analysis on a somewhat complicated form of information, i.e., memes. We propose a multi-task, multi-modal deep learning framework to solve multiple tasks si- multaneously. For multi-tasking, we propose two attention-like mechanisms viz., Inter-task Relationship Module (iTRM) and Inter-class Relationship Module (iCRM). The main mo- tivation of iTRM is to learn the relationship between the tasks to realize how they help each other. In contrast, iCRM develops rela- tions between the different classes of tasks. Fi- nally, representations from both the attentions are concatenated and shared across the five tasks (i.e., humour, sarcasm, offensive, moti- vational, and sentiment) for multi-tasking. We use the recently released dataset in the Mem- otion Analysis task @ SemEval 2020, which consists of memes annotated for the classes as mentioned above. Empirical results on Mem- otion dataset show the efficacy of our pro- posed approach over the existing state-of-the- art systems (Baseline and SemEval 2020 win- ner). The evaluation also indicates that the pro- posed multi-task framework yields better per- formance over the single-task learning. 1 Introduction The content and form of content shared on online social media platforms have changed rapidly over time. Currently, one of the most popular forms of media shared on such platforms is ’Memes’. Ac- cording to its definition from Oxford Dictionary, a meme is a piece of data, often in the form of im- ages, text or videos that carry cultural information through an imitable phenomenon with a mimicked theme, that is shared (sometimes with slight modi- fication) rapidly by internet users. Every meme can be associated with five affect values, namely humour (Hu), sarcastic (Sar), of- fensive (Off), motivational (Mo), and sentiment (Sent). Hence, in a broad sense, memes can be cat- egorized into four intersecting sets viz. humorous memes, sarcastic memes, offensive memes, and motivational memes. Humour refers to the quality of being amusing or comic. Formally, humour is defined as the na- ture of experiences to induce laughter and provide amusement. Humourous memes are the most pop- ular and widely used on social media platforms. An example for humourous memes is shown in Figure 1a. Sarcasm is often used to convey thinly veiled disapproval humorously. A sarcastic meme is a meme where an incongruity exists between the in- tended meaning and the way it is expressed. These are generally used …

Content truncated to 500 words