MemoSen: A Multimodal Dataset for Sentiment Analysis of Memes

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 1542–1554 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 1542 MemoSen: A Multimodal Dataset for Sentiment Analysis of Memes Eftekhar Hossain�, Omar SharifΨ, Mohammed Moshiul HoqueΨ �Department of Electronics and Telecommunication Engineering ΨDepartment of Computer Science and Engineering �ΨChittagong University of Engineering and Technology, Chattogram-4349, Bangladesh {eftekhar.hossain, omar.sharif, moshiul 240}@cuet.ac.bd Abstract Posting and sharing memes have become a powerful expedient of expressing opinions on social media in recent days. Analysis of sentiment from memes has gained much attention to researchers due to its substantial implications in various domains like finance and politics. Past studies on sentiment analysis of memes have primarily been conducted in English, where low-resource languages gain little or no attention. However, due to the proliferation of social media usage in recent years, sentiment analysis of memes has become a crucial research issue in low resource languages. The scarcity of benchmark dataset is a significant barrier in performing multimodal sentiment analysis research in resource-constrained languages like Bengali. This paper presents a novel multimodal dataset (named MemoSen) for Bengali containing 4368 memes with three annotated sentiment labels positive, negative, and neutral. A detailed annotation guideline is provided to facilitate further resource development in this domain. Additionally, a set of experiments are carried out on MemoSen by constructing twelve unimodal (i.e., visual, textual) and ten multimodal (image+text) models. The evaluation exhibits that the integration of multimodal information significantly improves (about 1.2%) the meme sentiment classification compared to the unimodal counterparts and thus elucidate the novel aspects of multimodality. Keywords: Sentiment analysis, Multimodal fusion, Memes, Code-mixing, Low resource languages 1. Introduction Recently, the usage of social media platforms (i.e. Facebook, Twitter, Instagram) has increased dramati- cally due to the substantial evolution of the Internet and various web 2.0 applications. These platforms have become a place where people share their opinions concerning business, politics, services, entertainment, and other current affairs. Automatic sentiment analysis of these conversations has grabbed increased attention from the NLP researchers in recent years since it helps to identify a user’s viewpoint or expression on a partic- ular event or topic (Hossain et al., 2020b; Bakliwal et al., 2013). To date, most of the researches have tried to classify the textual sentiment into three classes: posi- tive, negative and neutral classes (Mamta et al., 2020; Mamun et al., 2022). However, the mode of informa- tion in social media platforms is dramatically trans- forming day by day. The recent surge in multimodal (i.e., combination of image, text, and videos) content in these platforms has brought a new direction in sen- timent analysis research. One such multimodal content is the meme which has become a popular form of prop- agating information. Few pieces of researches (Pranesh and Shekhar, 2020; Wali´nska and Potoniec, 2020) have been conducted to analyze memes’ sentiment in En- glish by combining visual and textual features. Joint evaluation of features of multiple modalities is crucial to accurately infer a meme’s …

Content truncated to 500 words