NLP_UIOWA at SemEval-2020 Task 8: You’re Not the Only One Cursed with Knowledge - Multi Branch Model Memotion Analysis

Proceedings of the 14th International Workshop on Semantic Evaluation, pages 891–900 Barcelona, Spain (Online), December 12, 2020. 891 NLP UIOWA at SemEval-2020 Task 8: You’re not the only one cursed with knowledge - Multi branch model memotion analysis Ingroj Shrestha and Jonathan Rusert Department of Computer Science University of Iowa Iowa City, IA, USA {ingroj-shrestha,jonathan-rusert}@uiowa.edu Abstract We propose hybrid models (HybridE and HybridW) for meme analysis (SemEval 2020 Task 8), which involves sentiment classification (Subtask A), humor classification (Subtask B), and scale of semantic classes (Subtask C). The hybrid model consists of BLSTM and CNN for text and im- age processing respectively. HybridE provides equal weight to BLSTM and CNN performance, while HybridW provides weightage based on the performance of BLSTM and CNN on a valida- tion set. The performances (macro F1) of our hybrid model on Subtask A are 0.329 (HybridE), 0.328 (HybridW), on Subtask B are 0.507 (HybridE), 0.512 (HybridW), and on Subtask C are 0.309 (HybridE), 0.311 (HybridW). 1 Introduction Background. With the increasing social media culture, the sharing of internet memes on social media platforms has grown immensely in the recent years. Meme is defined as the unit of cultural information that replicates and transmits with reliability and fecundity (Linxia and Ziran, 2006). Memes are gener- ally an image paired with text, and used to express an array of ideas (e.g. humor, sarcasm). Memes can be derived from pop cultures, previous experiences, or even more abstract ideas. Memes have become a large part of internet culture, and can preserve viewpoints specific to the community from where it originated. Memes can be used to express humor, embarrassment, hate, and even more emotions. The creativity of memes, however, carry a downside. Hateful or offensive memes can also be created and can lead to an increase in hate crimes (Heikkil¨a, 2017; Sabat et al., 2019). As with hateful language, sev- eral social media platforms have been working on policies to control such hateful and offensive memes while being careful not to hinder the creativity of users’ expressions through memes (Kastrenakes, 2019; Hutchinson, 2020; Heilweil, 2020). One of the major steps in controlling the sharing of hateful memes is being able to successfully detect them. Detection of offensive content on social media is an ongoing task. Current attempts at detecting offensive memes is limited. Furthermore, detecting offensive memes is more challenging than detecting offensive text as it involves both visual and language understanding while the latter only requires lan- guage understanding. Currently, many sites rely on human moderators to identify and remove memes that express emotions that violate the platform’s policy. However, with the increasing use of memes across social media platforms, handpicking offensive memes would require larger human resource and can cause problems in scalability. Automated systems to identify the emotion of a meme could help in a first line defense/analysis of memes and could help reduce the load on human moderators. We already see this hybrid approach being employed for offensive and hateful text detection on several social …

Content truncated to 500 words