Developing a Dataset for Evaluating Approaches for Document Expansion with Images

Developing a Dataset for Evaluating Approaches for Document Expansion with Images Debasis Ganguly, Iacer Calixto, Gareth J.F. Jones ADAPT Centre, School of Computing, Dublin City University, Ireland {icalixto, dganguly, gjones}@computing.dcu.ie Abstract Motivated by the adage that a “picture is worth a thousand words” it can be reasoned that automatically enriching the textual content of a document with relevant images can increase the readability of a document. Moreover, features extracted from the additional image data inserted into the textual content of a document may, in principle, be also be used by a retrieval engine to better match the topic of a document with that of a given query. In this paper, we describe our approach of building a ground truth dataset to enable further research into automatic addition of relevant images to text documents. The dataset is comprised of the official ImageCLEF 2010 collection (a collection of images with textual metadata) to serve as the images available for automatic enrichment of text, a set of 25 benchmark documents that are to be enriched, which in this case are children’s short stories, and a set of manually judged relevant images for each query story obtained by the standard procedure of depth pooling. We use this benchmark dataset to evaluate the effectiveness of standard information retrieval methods as simple baselines for this task. The results indicate that using the whole story as a weighted query, where the weight of each query term is its tf-idf value, achieves an precision of 0.1714 within the top 5 retrieved images on an average. Keywords: Image Retrieval, Document Augmentation with Images 1. Introduction Document expansion, in addition to inserting text and hy- perlinks, can also involve adding non textual content such as images that are topically related to document text, in or- der to enhance the readability (or in fact indexability) of the text. For example, in (Hall et al., 2012), Wikipedia arti- cles are augmented with images retrieved from the Kirklees image archive, where automatically extracted key concepts from the Wiki text passages were used to formulate the queries for retrieving the images. The aim of our work, reported in this paper, is to build up a dataset for evaluation of the effectiveness of automated ap- proaches to document expansion with images. In particular, the problem that we address in this paper is that of augment- ing the text of children’s short stories (e.g. fairy tales and fables) with images in order to help improve the readability of the stories for small children according to the adage that “a picture is worth a thousand words”1. The “document ex- pansion with images” methodologies, developed and eval- uated on this dataset, can also be applied to augment other types of text documents, such as news articles, blogs etc. The illustration of children’s stories is a particular instance of the general problem of automatic text illustration, an in- herently multimodal problem that involves image process- ing and natural language processing. A related problem to automatic text illustration is that …

Content truncated to 500 words