SIS@IIITH at SemEval-2020 Task 8: An Overview of Simple Text Classification Methods for Meme Analysis

Proceedings of the 14th International Workshop on Semantic Evaluation, pages 1190–1194 Barcelona, Spain (Online), December 12, 2020. 1190 SIS@IIITH at SemEval-2020 Task 8: An Overview of Simple Text Classification Methods for Meme Analysis Sravani Boinepelli, Manish Shrivastava, Vasudeva Varma IIIT Hyderabad, India sravani.boinepelli@research.iiit.ac.in {m.shrivastava, vv}@iiit.ac.in Abstract Memes are steadily taking over the feeds of the public on social media. There is always the threat of malicious users on the internet posting offensive content, even through memes. Hence, the automatic detection of offensive images/memes is imperative along with detection of offensive text. However, this is a much more complex task as it involves both visual cues as well as language understanding and cultural/context knowledge. This paper describes our approach to the task of SemEval-2020 Task 8: Memotion Analysis. We chose to participate only in Task A which dealt with Sentiment Classification, which we formulated as a text classification problem. Through our experiments, we explored multiple training models to evaluate the performance of simple text classification algorithms on the raw text obtained after running OCR on meme images. Our submitted model achieved an accuracy of 72.69% and exceeded the existing baseline’s Macro F1 score by 8% on the official test dataset. Apart from describing our official submission, we shall elucidate how different classification models respond to this task. 1 Introduction Sentiment analysis of online social media has become an increasingly hot topic for researchers and has a wide range of applications from e-commerce, analyzing movie reviews, analysis of current trends, cam- paigns, etc. Understanding the sentiment bound to a phrase or sentence helps us discern the opinion of a person and a majority of this information is found on online social media such as Instagram, Face- book and Twitter. In concurrence with this, memes shared online are becoming an increasingly popular method for people to express themselves and their opinions, often in the form of a seemingly humorous visual image, on social media. According to Merriam-Webster Online (2020), a meme is an idea, behav- ior, or style that spreads by means of imitation from person to person within a culture and often carries symbolic meaning representing a particular phenomenon or theme. They are often shared on social me- dia in the form of an image or even a video. This phenomenon poses a great challenge to conventional NLP systems, which are generally purely text based. A hybrid approach is needed to tackle the problem of analysing the sentiment of a meme. It is therefore becoming exceedingly apparent for both the fields of NLP and Computer Vision to rise up to the challenge of adopting multi modal approaches as memes, videos, and other multi modal content are becoming increasingly prevalent online. Through our submis- sion, we aimed to perceive how basic machine learning models performed in this problem space. Our text classification model achieved a Macro-F1 score of 29.65% on the task leaderboard, which exceeded the existing baseline by 8% and ranked 29th among the 566 teams registered which is significant given the …

Content truncated to 500 words