Automatic Image Annotation Using Maximum Entropy Model

R. Dale et al. (Eds.): IJCNLP 2005, LNAI 3651, pp. 34 – 45, 2005. © Springer-Verlag Berlin Heidelberg 2005 Automatic Image Annotation Using Maximum Entropy Model Wei Li and Maosong Sun State Key Lab of Intelligent Technology and Systems, Department of Computer Science and Technology, Tsinghua University, Beijing 100084, China wei.lee04@gmail.com, sms@mail.tsinghua.edu.cn Abstract. Automatic image annotation is a newly developed and promising technique to provide semantic image retrieval via text descriptions. It concerns a process of automatically labeling the image contents with a pre-defined set of keywords which are exploited to represent the image semantics. A Maximum Entropy Model-based approach to the task of automatic image annotation is proposed in this paper. In the phase of training, a basic visual vocabulary con- sisting of blob-tokens to describe the image content is generated at first; then the statistical relationship is modeled between the blob-tokens and keywords by a Maximum Entropy Model constructed from the training set of labeled images. In the phase of annotation, for an unlabeled image, the most likely associated keywords are predicted in terms of the blob-token set extracted from the given image. We carried out experiments on a medium-sized image collection with about 5000 images from Corel Photo CDs. The experimental results demon- strated that the annotation performance of this method outperforms some tradi- tional annotation methods by about 8% in mean precision, showing a potential of the Maximum Entropy Model in the task of automatic image annotation. 1 Introduction Last decade has witnessed an explosive growth of multimedia information such as images and videos. However, we can’t access to or make use of the relevant informa- tion more leisurely unless it is organized so as to provide efficient browsing and que- rying. As a result, an important functionality of next generation multimedia informa- tion management system will undoubtedly be the search and retrieval of images and videos on the basis of visual content. In order to fulfill this “intelligent” multimedia search engines on the world-wide- web, content-based image retrieval techniques have been studied intensively during the past few years. Through the sustained efforts, a variety of state-of-the-art methods employing the query-by-example (QBE) paradigm have been well established. By this we mean that queries are images and the targets are also images. In this manner, vis- ual similarity is computed between user-provided image and database images based on the low-level visual features such as color, texture, shape and spatial relationships. However, two important problems still remain. First, due to the limitation of object recognition and image understanding, semantics-based image segmentation algorithm Automatic Image Annotation Using Maximum Entropy Model 35 is unavailable, so segmented region may not correspond to users’ query object. Sec- ond, visual similarity is not semantic similarity which means that low-level features are easily extracted and measured, but from the users’ point of view, they are non- intuitive. It is not easy to use them to formulate the user’s needs. We encounter a so- called semantic gap here. Typically the starting point of the retrieval …

Content truncated to 500 words