N24News: A New Dataset for Multimodal News Classification

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 6768–6775 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 6768 N24News: A New Dataset for Multimodal News Classification Zhen Wang1∗, Xu Shan2∗, Xiangxie Zhang3, Jie Yang4 Delft University of Technology, Netherlands {1z.wang-42, 3x.zhang-60}@student.tudelft.nl, {2x.shan-2, 4j.yang-3}@tudelft.nl Abstract Current news datasets merely focus on text features on the news and rarely leverage the feature of images, excluding numerous essential features for news classification. In this paper, we propose a new dataset, N24News, which is generated from New York Times with 24 categories and contains both text and image information in each news. We use a multitask multimodal method and the experimental results show multimodal news classification performs better than text-only news classification. Depending on the length of the text, the classification accuracy can be increased by up to 8.11%. Our research reveals the relationship between the performance of a multimodal classifier and its sub-classifiers, and also the possible improvements when applying multimodal in news classification. N24News is shown to have great potential to prompt the multimodal news studies. Keywords: Multimodal Dataset, News Article, Text Classification 1. Introduction People have tried to use different carriers to record news. Ancient people first drew images by hands-on walls to record important things. After language was invented, words became the main tools for recording. Thanks to parchment preserved to this day, we can study people who lived a long time ago. Later, with the invention of the camera, images are widely used in news. Compared with text, images can bring us more intuitive information, even if we cannot understand the language used in the news. It is safe to say that images and text play an equally important role in news. News classification is one of the essential tasks in news research (Katari and Myneni, 2020). We use the infor- mation provided by the news to group them into dif- ferent categories.There is already some research about news classification, for example, news datasets, such as 20NEWS (Lang, 1995) and AG News (Zhang et al., 2015). However, they choose to ignore the images and merely pay attention to the text. This is not in line with the actual situation, especially when almost all the news today has images. In this work, we aim to use both images and text to achieve better news classifica- tion. In order to combine heterogeneous information ex- tracted from images and texts, multimodal methods are needed. Multimodal approaches can process var- ious types of information simultaneously and has been used in news studies before. For example, in the fake news dataset Fakeddit (Nakamura et al., 2019), the au- thors propose a hybrid text+image model to classifier fake news. However, to best of our knowledge, cur- rently there is no valid public news dataset containing enough real news with both images and texts that can be used to do multimodal news classification. Thus, in this work, we use the New York Times to build a …

Content truncated to 500 words