The Use of Monolingual Context Vectors for Missing Translations in Cross-Language Information Retrieval

R. Dale et al. (Eds.): IJCNLP 2005, LNAI 3651, pp. 22 – 33, 2005. © Springer-Verlag Berlin Heidelberg 2005 The Use of Monolingual Context Vectors for Missing Translations in Cross-Language Information Retrieval Yan Qu1, Gregory Grefenstette2, and David A. Evans1 1 Clairvoyance Corporation, 5001 Baum Boulevard, Suite 700, Pittsburgh, PA, 15213, USA {yqu, dae}@clairvoyancecorp.com 2 LIC2M/SCRI/LIST/DTSI/CEA, B.P.6, 92265 Fontenay-aux-Roses Cedex, France {Gregory.Grefenstette}@cea.fr Abstract. For cross-language text retrieval systems that rely on bilingual dic- tionaries for bridging the language gap between the source query language and the target document language, good bilingual dictionary coverage is imperative. For terms with missing translations, most systems employ some approaches for expanding the existing translation dictionaries. In this paper, instead of lexicon expansion, we explore whether using the context of the unknown terms can help mitigate the loss of meaning due to missing translation. Our approaches consist of two steps: (1) to identify terms that are closely associated with the unknown source language terms as context vectors and (2) to use the translations of the associated terms in the context vectors as the surrogate translations of the un- known terms. We describe a query-independent version and a query-dependent version using such monolingual context vectors. These methods are evaluated in Japanese-to-English retrieval using the NTCIR-3 topics and data sets. Em- pirical results show that both methods improved CLIR performance for short and medium-length queries and that the query-dependent context vectors per- formed better than the query-independent versions. 1 Introduction For cross-language text retrieval systems that rely on bilingual dictionaries for bridg- ing the language gap between the source query language and the target document language, good bilingual dictionary coverage is imperative [8,9]. Yet, translations for proper names and special terminology are often missing in available dictionaries. Various methods have been proposed for finding translations of names and terminol- ogy through transliteration [5,11,13,14,16,18,20] and corpus mining [6,7,12,15,22]. In this paper, instead of attempting to find the candidate translations of terms without translations to expand existing translation dictionaries, we explore to what extent simply using text context can help mitigate the missing translation problem and for what kinds of queries. The context-oriented approaches include (1) identifying words that are closely associated with the unknown source language terms as context vectors and (2) using the translations of the associated words in the context vectors as the surrogate translations of the unknown words. We describe a query-independent The Use of Monolingual Context Vectors for Missing Translations 23 version and a query-dependent version using such context vectors. We evaluate these methods in Japanese-to-English retrieval using the NTCIR-3 topics and data sets. In particular, we explore the following questions: • Can translations obtained from context vectors help CLIR performance? • Are query-dependent context vectors more effective than query-independent context vectors for CLIR? In the balance of this paper, we first describe related work in Section 2. The methods of obtaining translations through context vectors are presented in Section 3. The CLIR evaluation system and evaluation results are presented in Section 4 and …

Content truncated to 500 words