Cross-Language Multimedia Information Retrieval

Abstract Cross-Language Multimedia Information Retrieval Sharon Flank emotion, Inc. 2600 Park Tower Dr., Vienna, VA 22180 USA sharon.flank@emotion.com Simple measures can achieve high-accuracy cross-language retrieval in carefully chosen applications. Image retrieval is one of those applications, with results ranging from 68% of human translator performance for German, to 100% for French. 1 Introduction contain strings of keywords. Typical queries are, as in most Web search applications, two to three words in length. At this point, all of the captions are in English. eMotion hosts a large database of images for sale and for licensing, PictureQuest. At least 10% of PictureQuest's user base is outside the United States. The tests were performed on the PictureQuest database of approximately 400,000 images. Information is increasingly global, and the need to access it crosses language barriers. The topic of this paper, cross-language information retrieval, concerns the automatic retrieval of text in one language via a query in a different language. A considerable body of literature has grown up around cross-language information retrieval (e.g. Grefenstette 1998, TREC-7 1999). There are two basic approaches. Either the query can be translated, or each entire document can be translated into the same language as the query. The accuracy of retrieval across languages, however, is generally not good. One of the weaknesses that plagues cross- language retrieval is that we do not have a good sense of who the users are, or how best to interact with them. In this paper we describe a multimedia application for which cross-language information retrieval works particularly well. eMotion, Inc. has developed a natural language information retrieval application that retrieves images, such as photographs, based on short textual descriptions or captions. The captions are typically one to three sentences, although they may also Recent Web utilization data for PictureQuest indicate that of the 10% of users from outside the United States, a significant portion come from Spanish-speaking, French-speaking, and German-speaking countries. It is expected that adding appropriate language interfaces and listing PictureQuest in foreign-language search engines will dramatically increase non- English usage. The Cross-Language Multimedia Retrieval Application This paper offers several original contributions to the literature on cross- language information retrieval. First, the choice of application is novel, and significant because it simplifies the language problem enough to make it tractable. Because the objects retrieved are images and not text, they are instantly comprehensible to the user regardless of language issues. This fact makes it possible for users to perform a relevance assessment without the need for any kind of translation. More important, users themselves can select objects of interest, without recourse to translation. The images are, in fact, 13 associated with caption information, but, even in the monolingual system, few users ever even view the captions. It should be noted that most of the images in PictureQuest are utilized for advertising and publishing, rather than for news applications. Users of history and news photos do tend to check the captions, and often users in publishing will view the captions. For advertising, however, …

Content truncated to 500 words