NIT-Agartala-NLP-Team at SemEval-2020 Task 8: Building Multimodal Classifiers to Tackle Internet Humor

Proceedings of the 14th International Workshop on Semantic Evaluation, pages 1179–1189 Barcelona, Spain (Online), December 12, 2020. 1179 NIT-Agartala-NLP-Team at SemEval-2020 Task 8: Building Multimodal Classifiers to tackle Internet Humor Steve Durairaj Swamy Shubham Laddha Basil Abdussalam Debayan Datta Anupam Jamatia Department of Computer Science and Engineering National Institute of Technology Agartala, Tripura, India {steve050798, laddha.shubham97, basilwkrh, debayan.datta98, anupamjamatia}@gmail.com Abstract The paper describes the systems submitted to SemEval-2020 Task 8: Memotion by the ‘NIT- Agartala-NLP-Team’. A dataset of 8879 memes was made available by the task organizers to train and test our models. Our systems include a Logistic Regression baseline, a BiLSTM + Attention-based learner and a transfer learning approach with BERT. For the three sub-tasks A, B and C, we attained ranks 24/33, 11/29 and 15/26, respectively. We highlight our difficulties in harnessing image information as well as some techniques and handcrafted features we employ to overcome these issues. We also discuss various modelling issues and theorize possible solutions and reasons as to why these problems persist. 1 Introduction Over the years, the internet and social media have become an indispensable part of our lives. Today, an average netizen spends over 45 minutes on some form of social media everyday1. Therefore, social media has become a goldmine for data to model and study human opinions and behaviour. Social media analysis conventionally deals with data in the form of text, audio or video — but often only one prominent modality is studied in isolation. This lack of multimodality in research works has led to entire modes of communication being disregarded. One such form of internet communication that we have yet to tap into is memes. While a meme was initially defined as “an idea, behaviour, or style that spreads from person to person within a culture—often to convey a particular phenomenon, theme, or meaning2,” it has transformed into the umbrella term for a suite of referential humor that plagues the internet. Memes can more aptly be defined as “a form of referential humor that incites humor by leveraging images, text and sometimes audio.” The image is more often than not coupled with a real-life event or media. The images are then re-purposed to incite humor through the subversion of expectations and via reference. Through the years, memes have been leveraged differently by various communities — apart from innocent, humor inducing purposes. Corporations and companies are increasingly interested in harnessing memes to sway their younger customers. More adversely, certain memes such as Pepe the Frog3 were adopted by alt-right groups and used as a calling card of sorts. Memes are also used as performative acts, which involve a conscious decision to support or oppose a movement (as seen in the case of the recent Hong Kong protests 4). The abundance of memes on social media platforms such as Facebook, Instagram, and Twitter (Zannettou et al., 2018b) further suggests that — these digital constructs have become a flagship of internet culture, so and so that, to understand memes would mean to understand …

Content truncated to 500 words