A Corpus of Images and Text in Online News

A Corpus of Images and Text in Online News Laura Hollink∗, Adriatik Bedjeti∗, Martin van Harmelen∗ and Desmond Elliott∗† ∗Centrum Wiskunde & Informatica (CWI), Amsterdam, The Netherlands †ILLC, University of Amsterdam, Amsterdam, The Netherlands Email: {l.hollink, a.bedjeti}@cwi.nl, martin@vanharmelen.com, d.elliott@uva.nl Abstract In recent years, several datasets have been released that include images and text, giving impulse to new methods that combine natural language processing and computer vision. However, there is a need for datasets of images in their natural textual context. The ION corpus contains 300K news articles published between August 2014 - 2015 in five online newspapers from two countries. The 1-year coverage over multiple publishers ensures a broad scope in terms of topics, image quality and editorial viewpoints. The corpus consists of JSON-LD files with the following data about each article: the original URL of the article on the news publisher’s website, the date of publication, the headline of the article, the URL of the image displayed with the article (if any), and the caption of that image. Neither the article text nor the images themselves are included in the corpus. Instead, the images are distributed as high-dimensional feature vectors extracted from a Convolutional Neural Network, anticipating their use in computer vision tasks. The article text is represented as a list of automatically generated entity and topic annotations in the form of Wikipedia/DBpedia pages. This facilitates the selection of subsets of the corpus for separate analysis or evaluation. Keywords: online news; image features; topic extraction 1. Introduction In recent years, several datasets have been released that in- clude images and text (Ferraro et al., 2015; Bernardi et al., 2016), giving impulse to new methods that combine natu- ral language processing and computer vision, such as au- tomatic image description (Fang et al., 2014) and image– sentence matching (Hodosh et al., 2013). Current datasets typically consist of user-captioned images from Flickr1 (Ordonez et al., 2011; Chen et al., 2015) or images with descriptions produced by crowd workers (Rashtchian et al., 2010; Elliott and Keller, 2013; Zitnick and Parikh, 2013; Lin et al., 2014; Young et al., 2014). These datasets can be used to train systems to produce Flickr-like captions or crowdsourced descriptions, but we argue there is a need for more datasets for generating captions in context. This argument has previously been made by Feng and Lapata (2008), who released a dataset of 3,361 news articles from BBC News, including images and real-world captions; Tir- illy et al. (2010) collected the texts, images and captions of 27,000 French newspaper articles, and used the real-world captions as a ground truth to evaluate an image annotation algorithm. As a continuation of these efforts, we present the Images in Online News (ION) corpus. The ION corpus contains news articles published between August 2014 - 2015 in five online newspapers from two different countries. It includes more than 323,707 articles, making it larger than the existing datasets of news images and text. The 1-year coverage over multiple publishers en- sures a broad scope in …

Content truncated to 500 words