Mining New Word Translations from Comparable Corpora

Mining New Word Translations from Comparable Corpora Li Shao and Hwee Tou Ng Department of Computer Science National University of Singapore 3 Science Drive 2, Singapore 117543 {shaoli, nght}@comp.nus.edu.sg Abstract New words such as names, technical terms, etc appear frequently. As such, the bilingual lexicon of a machine translation system has to be constantly updated with these new word translations. Comparable corpora such as news documents of the same period from dif- ferent news agencies are readily available. In this paper, we present a new approach to min- ing new word translations from comparable corpora, by using context information to complement transliteration information. We evaluated our approach on six months of Chi- nese and English Gigaword corpora, with en- couraging results. 1. Introduction New words such as person names, organization names, technical terms, etc. appear frequently. In order for a machine translation system to trans- late these new words correctly, its bilingual lexi- con needs to be constantly updated with new word translations. Much research has been done on using parallel corpora to learn bilingual lexicons (Melamed, 1997; Moore, 2003). But parallel corpora are scarce resources, especially for uncommon lan- guage pairs. Comparable corpora refer to texts that are not direct translation but are about the same topic. For example, various news agencies report major world events in different languages, and such news documents form a readily avail- able source of comparable corpora. Being more readily available, comparable corpora are thus more suitable than parallel corpora for the task of acquiring new word translations, although rela- tively less research has been done in the past on comparable corpora. Previous research efforts on acquiring translations from comparable corpora include (Fung and Yee, 1998; Rapp, 1995; Rapp, 1999). When translating a word w, two sources of in- formation can be used to determine its transla- tion: the word w itself and the surrounding words in the neighborhood (i.e., the context) of w. Most previous research only considers one of the two sources of information, but not both. For exam- ple, the work of (Al-Onaizan and Knight, 2002a; Al-Onaizan and Knight, 2002b; Knight and Graehl, 1998) used the pronunciation of w in translation. On the other hand, the work of (Cao and Li, 2002; Fung and Yee, 1998; Koehn and Knight, 2002; Rapp, 1995; Rapp, 1999) used the context of w to locate its translation in a second language. In this paper, we propose a new approach for the task of mining new word translations from comparable corpora, by combining both context and transliteration information. Since both sources of information are complementary, the accuracy of our combined approach is better than the accuracy of using just context or translitera- tion information alone. We fully implemented our method and tested it on Chinese-English comparable corpora. We translated Chinese words into English. That is, Chinese is the source language and English is the target language. We achieved encouraging results. While we have only tested our method on Chi- nese-English comparable corpora, our method is general …

Content truncated to 500 words