NUAA-QMUL at SemEval-2020 Task 8: Utilizing BERT and DenseNet for Internet Meme Emotion Analysis

Proceedings of the 14th International Workshop on Semantic Evaluation, pages 901–907 Barcelona, Spain (Online), December 12, 2020. 901 NUAA-QMUL at SemEval-2020 Task 8: Utilizing BERT and DenseNet for Internet Meme Emotion Analysis Xiaoyu Guo Nanjing University of Aeronautics and Astronautics xy deepa@163.com Jing Ma Nanjing University of Aeronautics and Astronautics majing5525@126.com Arkaitz Zubiaga Queen Mary University of London a.zubiaga@qmul.ac.uk Abstract This paper describes our contribution to SemEval 2020 Task 8: Memotion Analysis. Our system learns multi-modal embeddings from text and images in order to classify Internet memes by sentiment. Our model learns text embeddings using BERT and extracts features from images with DenseNet, subsequently combining both features through concatenation. We also compare our results with those produced by DenseNet, ResNet, BERT, and BERT-ResNet. Our results show that image classification models have the potential to help classifying memes, with DenseNet outperforming ResNet. Adding text features is however not always helpful for Memotion Analysis. 1 Introduction The growing ubiquity of Internet memes on social media platforms such as Facebook, Instagram, and Twitter reinforces the importance of processing multimodal content. Despite memes being so deeply ingrained in the Internet culture, it is still hard to leverage AI methods to understand the underlying meanings expressed by memes. This, in turn, makes it challenging to identify certain kinds of content in these memes, such as malicious information posted on social media. To tackle this problem, SemEval 2020 Task 8 held a meme emotion analysis task(Sharma et al., 2020). It consisted of three sub-tasks: (1) predicting the sentiment of Internet memes as positive, neutral or negative, (2) identifying the type of humor expressed as sarcastic, humorous, offensive or motivation, and (3) quantifying the extent to which a particular emotion is being expressed. Internet memes can comprise a variety of modalities such as textual, visual, and audio. Adhering to the task guidelines at hand, we approach the problem by considering only the image-with-caption class of meme, which is commonly found on social media. This paper describes a system that combines BERT (Bidirectional Encoder Representation from Transformers) and DenseNet (Dense Convolutional Network) to classify Internet memes. We find that the performance of Multi-modal Deep Neural Network is better than one model Deep Neural Network and the performance of ResNet (Residual Network) is worse than DenseNet. This paper is organized as follows. In section Background, we describe Memotion Analysis task, the characteristics of the Internet memes, prior work on multi-modal classification, prior research exploring data analysis approaches to study memes, and our contributions in this paper. Then in Section 3, we propose BERT-DenseNet, a deep learning model to classify memes. We then use the models to study memes in task A, task B, and task C in Section 4. In Section 5, we display the results of our experiments, including comparative experiments. Finally, we conclude the findings of this research and suggest directions for future work. This work is licensed under a Creative Commons Attribution 4.0 International Licence. Licence details: http:// creativecommons.org/licenses/by/4.0/. Source code for our model and for …

Content truncated to 500 words