UI at SemEval-2020 Task 8: Text-Image Fusion for Sentiment Classification

Proceedings of the 14th International Workshop on Semantic Evaluation, pages 1195–1200 Barcelona, Spain (Online), December 12, 2020. 1195 UI at SemEval-2020 Task 8: Text-Image Fusion for Sentiment Classification Andi Suciati Universitas Indonesia Depok, Indonesia andi.suciati@ui.ac.id Indra Budi Universitas Indonesia Depok, Indonesia indra@cs.ui.ac.id Abstract This paper describes our system, UI, for task A: Sentiment Classification in SemEval-2020 Task 8 Memotion Analysis. We use a common traditional machine learning, which is SVM, by utilizing the combination of text and images features. The data consist text that extracted from memes and the images of memes. We employ n-gram language model for text features and pre-trained model, VGG-16, for image features. After obtaining both features from text and images in form of 2-dimensional arrays, we concatenate and classify the final features using SVM. The experiment results show SVM achieved 35% for its F1 macro, which is 0.132 points or 13.2% above the baseline model. 1 Introduction SemEval-2020 Task 8 Memotion Analysis consists three tasks, which are Sentiment Classification, Humor Classification, and Scales of Semantic Classes1. For this research, we chose to do the task A because we have interest in sentiment analysis area. In task A: Sentiment Classification, we need to classify the sentiment of given English memes to three sentiment categories, which are positive, negative, and neutral. Sentiment analysis is one of NLP task that conducted for extracting the sentiment, emotions, or judgement of reviews and classified it. Mainly, sentiment analysis task requires dataset in form of text. However, the opinion that people express is not only in form of text, such as comments, reviews, but also in other form, for example, image with text, or can be called memes. According to Davidson (2012), an internet meme is a piece of culture, typically a joke, which gains influence through online transmission. In social media, usually the memes are expressed in images that contain short text which express the author opinion towards object. This makes classifying the sentiment of memes a challenge since the opinion in memes can be expressed explicit or implicitly. In this study, we conducted sentiment classification task on memes by combining the features from images and text, then classified it as positive, negative, or neutral. By using memes as data, it is believed that we can obtained better sentiment classification results because the other data source such as images, can enhance the robustness of models (Yu and Jiang, 2019). The data that used were obtained from SemEval 2020 Task 8: Memotion Analysis (Sharma et al., 2020). As we want to see how traditional machine learning works in this case of classification, we compared four machine learning algorithms and a neural network. Our system shows that it can achieve high scores in the testing set compared to the baseline scores even though the score was still considered low score. The rest of this paper is arranged as follows: In section 2, we review the related works with our study. Section 3 talks about the data that we used, including training and …

Content truncated to 500 words