MINT: A Method for Effective and Scalable Mining of Named Entity Transliterations from Large Comparable Corpora

Proceedings of the 12th Conference of the European Chapter of the ACL, pages 799–807, Athens, Greece, 30 March – 3 April 2009. c⃝2009 Association for Computational Linguistics MINT: A Method for Effective and Scalable Mining of Named Entity Transliterations from Large Comparable Corpora Raghavendra Udupa K Saravanan A Kumaran Jagadeesh Jagarlamudi* Microsoft Research India Bangalore 560080 INDIA [raghavu,v-sarak,kumarana,jags}@microsoft.com Abstract In this paper, we address the problem of min- ing transliterations of Named Entities (NEs) from large comparable corpora. We leverage the empirical fact that multilingual news ar- ticles with similar news content are rich in Named Entity Transliteration Equivalents (NETEs). Our mining algorithm, MINT, uses a cross-language document similarity model to align multilingual news articles and then mines NETEs from the aligned articles using a transliteration similarity model. We show that our approach is highly effective on 6 different comparable corpora between English and 4 languages from 3 different language families. Furthermore, it performs substantially better than a state-of-the-art competitor. 1 Introduction Named Entities (NEs) play a critical role in many Natural Language Processing and Information Retrieval (IR) tasks. In Cross-Language Infor- mation Retrieval (CLIR) systems, they play an even more important role as the accuracy of their transliterations is shown to correlate highly with the performance of the CLIR systems (Mandl and Womser-Hacker, 2005, Xu and Weischedel, 2005). Traditional methods for transliterations have not proven to be very effective in CLIR. Machine Transliteration systems (AbdulJaleel and Larkey, 2003; Al-Onaizan and Knight, 2002; Virga and Khudanpur, 2003) usually produce incorrect transliterations and translation lexcions such as hand-crafted or statistical dictionaries are too static to have good coverage of NEs1 occur- ring in the current news events. Hence, there is a critical need for creating and continually updat- * Currently with University of Utah. 1 New NEs are introduced to the vocabulary of a lan- guage every day. On an average, 260 and 452 new NEs appeared daily in the XIE and AFE segments of the LDC English Gigaword corpora respectively. ing multilingual Named Entity transliteration lexicons. The ubiquitous availability of comparable news corpora in multiple languages suggests a promising alternative to Machine Transliteration, namely, the mining of Named Entity Translitera- tion Equivalents (NETEs) from such corpora. News stories are typically rich in NEs and there- fore, comparable news corpora can be expected to contain NETEs (Klementiev and Roth, 2006; Tao et al., 2006). The large quantity and the per- petual availability of news corpora in many of the world’s languages, make mining of NETEs a viable alternative to traditional approaches. It is this opportunity that we address in our work. In this paper, we detail an effective and scala- ble mining method, called MINT (MIning Named-entity Transliteration equivalents), for mining of NETEs from large comparable corpo- ra. MINT addresses several challenges in mining NETEs from large comparable corpora: exhaus- tiveness (in mining sparse NETEs), computa- tional efficiency (in scaling on corpora size), language independence (in being applicable to many language pairs) and linguistic frugality (in requiring minimal external linguistic resources). …

Content truncated to 500 words