Cross-lingual Name Tagging and Linking for 282 Languages

Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 1946–1958 Vancouver, Canada, July 30 - August 4, 2017. c⃝2017 Association for Computational Linguistics https://doi.org/10.18653/v1/P17-1178 Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 1946–1958 Vancouver, Canada, July 30 - August 4, 2017. c⃝2017 Association for Computational Linguistics https://doi.org/10.18653/v1/P17-1178 Cross-lingual Name Tagging and Linking for 282 Languages Xiaoman Pan1, Boliang Zhang1, Jonathan May2, Joel Nothman3, Kevin Knight2, Heng Ji1 1 Computer Science Department, Rensselaer Polytechnic Institute {panx2,zhangb8,jih}@rpi.edu 2 Information Sciences Institute, University of Southern California {jonmay,knight}@isi.edu 3 Sydney Informatics Hub, University of Sydney joel.nothman@gmail.com Abstract The ambitious goal of this work is to de- velop a cross-lingual name tagging and linking framework for 282 languages that exist in Wikipedia. Given a document in any of these languages, our framework is able to identify name mentions, as- sign a coarse-grained or fine-grained type to each mention, and link it to an En- glish Knowledge Base (KB) if it is link- able. We achieve this goal by perform- ing a series of new KB mining meth- ods: generating “silver-standard” annota- tions by transferring annotations from En- glish to other languages through cross- lingual links and KB properties, refining annotations through self-training and topic selection, deriving language-specific mor- phology features from anchor links, and mining word translation pairs from cross- lingual links. Both name tagging and link- ing results for 282 languages are promis- ing on Wikipedia data and on-Wikipedia data. All the data sets, resources and sys- tems for 282 languages are made publicly available as a new benchmark 1. 1 Introduction Information provided in languages which people can understand saves lives in crises. For exam- ple, language barrier was one of the main diffi- culties faced by humanitarian workers responding to the Ebola crisis in 2014. We propose to break language barriers by extracting information (e.g., entities) from a massive variety of languages and ground the information into an existing knowledge base which is accessible to a user in his/her own 1http://nlp.cs.rpi.edu/wikiann language (e.g., a reporter from the World Health Organization who speaks English only). Wikipedia is a massively multi-lingual resource that currently hosts 295 languages and contains naturally annotated markups 2 and rich informa- tional structures through crowd-sourcing for 35 million articles in 3 billion words. Name mentions in Wikipedia are often labeled as anchor links to their corresponding referent pages. Each entry in Wikipedia is also mapped to external knowl- edge bases such as DBpedia3, YAGO (Mahdis- oltani et al., 2015) and Freebase (Bollacker et al., 2008) that contain rich properties. Figure 1 shows an example of Wikipedia markups and KB prop- erties. We leverage these markups for develop- ✤ Wikipedia Article: Mao Zedong (d. 26 Aralık 1893 - ö. 9 Eylül 1976), Çinli devrimci ve siyasetçi. Çin Komünist Partisinin (ÇKP) ve Çin Halk Cumhuriyetinin kurucusu. ✤ Wikipedia Markup: [[Mao Zedong]] (d. [[26 Aralık]] [[1893]] - ö. [[9 Eylül]] [[1976]]), Çinli devrimci ve siyasetçi. [[Çin Komünist Partisi]]nin (ÇKP) ve [[Çin Halk Cumhuriyeti]]nin …

Content truncated to 500 words