Concept Unification of Terms in Different Languages for IR

Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the ACL, pages 641–648, Sydney, July 2006. c⃝2006 Association for Computational Linguistics Concept Unification of Terms in Different Languages for IR Qing Li, Sung-Hyon Myaeng Information & Communications University, Korea {liqing,myaeng}@icu.ac.kr Yun Jin Chungnam National University, Korea wkim@cnu.ac.kr Bo-yeong Kang Seoul National University, Korea comeng99@snu.ac.kr Abstract Due to the historical and cultural reasons, English phases, especially the proper nouns and new words, frequently appear in Web pages written primarily in Asian languages such as Chinese and Korean. Although these English terms and their equivalences in the Asian languages refer to the same concept, they are erroneously treated as independent index units in tra- ditional Information Retrieval (IR). This paper describes the degree to which the problem arises in IR and suggests a novel technique to solve it. Our method firstly extracts an English phrase from Asian language Web pages, and then unifies the extracted phrase and its equivalence(s) in the language as one index unit. Experi- mental results show that the high preci- sion of our conceptual unification ap- proach greatly improves the IR perform- ance. 1 Introduction The mixed use of English and local languages presents a classical problem of vocabulary mis- match in monolingual information retrieval (MIR). The problem is significant especially in Asian language because words in the local lan- guages are often mixed with English words. Al- though English terms and their equivalences in a local language refer to the same concept, they are erroneously treated as independent index units in traditional MIR. Such separation of semantically identical words in different languages may limit retrieval performance. For instance, as shown in Figure 1, there are three kinds of Chinese Web pages containing information related with “Viterbi Algorithm (韦特比算法)”. The first case contains “Viterbi Algorithm” but not its Chinese equivalence “韦特比算法”. The second Figure 1. Three Kinds of Web Pages contains “韦特比算法” but not “Viterbi Algo- rithm”. The third has both of them. A user would expect that a query with either “Viterbi Algo- rithm” or “韦特比算法” would retrieve all of these three groups of Chinese Web pages. Oth- erwise some potentially useful information will be ignored. Furthermore, one English term may have sev- eral corresponding terms in a different language. For instance, Korean words “디지탈”, “디지틀”, and “디지털” are found in local Web pages, which all correspond to the English word “digi- tal” but are in different forms because of differ- ent phonetic interpretations. Establishing an equivalence class among the three Korean words and the English counterpart is indispensable. By doing so, although the query is “디지탈”, the Web pages containing “디지틀”, “디지털” or “digital” can be all retrieved. The same goes to Chinese terms. For example, two same semantic Chinese terms “维特比” and “韦特比” corre- spond to one English term “Viterbi”. There should be a semantic equivalence relation be- tween them. Although tracing the original English term from a term in a native language by back trans- literation (Jeong et al., …

Content truncated to 500 words