PRHLT-UPV at SemEval-2020 Task 8: Study of Multimodal Techniques for Memes Analysis

Proceedings of the 14th International Workshop on Semantic Evaluation, pages 908–915 Barcelona, Spain (Online), December 12, 2020. 908 PRHLT-UPV at SemEval-2020 Task 8: Study of Multimodal Techniques for Memes Analysis Gretel Liz De la Pe˜na Sarrac´en and Paolo Rosso and Anastasia Giachanou Universitat Polit`ecnica de Val`encia, Val`encia, Spain gredela@posgrado.upv.es prosso@dsic.upv.es angia9@upv.es Abstract This paper describes the system submitted by the PRHLT-UPV team for the task 8 of SemEval- 2020: Memotion Analysis. We propose a multimodal model that combines pretrained models of the BERT and VGG architectures. The BERT model is used to process the textual information and VGG the images. The multimodal model is used to classify memes according to the presence of offensive, sarcastic, humorous and motivating content. Also, a sentiment analysis of memes is carried out with the proposed model. In the experiments, the model is compared with other approaches to analyze the relevance of the multimodal model. The results show encouraging performances on the final leaderboard of the competition, reaching good positions in the ranking of systems. 1 Introduction Nowadays, social networks have become one of the main means of messages spread. Shared information usually consists of different types of content. In this sense, we can find publications of videos, images, texts or audios. Memes are examples of posts that usually combine image and text. Daily the number of memes on popular sites like Twitter and Facebook increases considerably. This makes automatic memes analysis a very important task at present. Often, memes contain subjective information, and can have ironic or sarcastic content. Irony and sarcasm are forms of figurative speech in which authors write the opposite of what they mean (Hern´andez Farıas and Rosso, 2016). Irony is an umbrella term whereas sarcasm is considered by many researchers a more direct and sometimes somewhat aggressive form of irony. Therefore, the analysis of memes requires an extensive study that combines several tasks. On the one hand, it is necessary to design multimodal systems that can handle both textual and visual information. On the other hand, different studies should be carried out to obtain all the semantic information that the texts may contain. In this way, the study can involve sentiment analysis, irony or sarcasm detection, and so on. Offensive language detection is an important task among them. It has gained great interest with the growth of the spread of offenses on Internet and its implication in society. The Memotion Analysis shared task has been organized in order to bring the attention towards Internet memes processing. The three subtasks defined in the task are Sentiment Classification (subtask A), Humour Classification (subtask B) and Scales of Semantic Classes (subtask C). The main goal of the subtask A is to classify a meme as positive, negative or neutral according to sentiment content. The subtask B aims to identify the type of humour expressed among the categories: sarcasm, humour, offense and motivation. It should be noted that many researchers do not consider sarcasm as a humour category since irony/sarcasm partially overlap with humour …

Content truncated to 500 words