A Multi-media Approach to Cross-lingual Entity Knowledge Transfer

Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pages 54–65, Berlin, Germany, August 7-12, 2016. c⃝2016 Association for Computational Linguistics A Multi-media Approach to Cross-lingual Entity Knowledge Transfer Di Lu1, Xiaoman Pan1, Nima Pourdamghani2, Shih-Fu Chang3, Heng Ji1, Kevin Knight2 1 Computer Science Department, Rensselaer Polytechnic Institute {lud2,panx2,jih}@rpi.edu 2 Information Sciences Institute, University of Southern California {damghani,knight}@isi.edu 3 Electrical Engineering Department, Columbia University sfchang@ee.columbia.edu Abstract When a large-scale incident or disaster oc- curs, there is often a great demand for rapidly developing a system to extract detailed and new information from low- resource languages (LLs). We propose a novel approach to discover compara- ble documents in high-resource languages (HLs), and project Entity Discovery and Linking results from HLs documents back to LLs. We leverage a wide variety of language-independent forms from multi- ple data modalities, including image pro- cessing (image-to-image retrieval, visual similarity and face recognition) and sound matching. We also propose novel meth- ods to learn entity priors from a large-scale HL corpus and knowledge base. Using Hausa and Chinese as the LLs and En- glish as the HL, experiments show that our approach achieves 36.1% higher Hausa name tagging F-score over a costly super- vised model, and 9.4% higher Chinese- to-English Entity Linking accuracy over state-of-the-art. 1 Introduction In many situations such as disease outbreaks and natural calamities, we often need to develop an In- formation Extraction (IE) component (e.g., a name tagger) within a very limited time to extract infor- mation from low-resource languages (LLs) (e.g., locations where Ebola outbreaks from Hausa doc- uments). The main challenge lies in the lack of la- beled data and linguistic processing tools in these languages. A potential solution is to extract and project knowledge from high-resource languages (HLs) to LLs. A large amount of non-parallel, domain-rich, topically-related comparable corpora naturally ex- ist across LLs and HLs for breaking incidents, such as coordinated news streams (Wang et al., 2007) and code-switching social media (Voss et al., 2014; Barman et al., 2014). However, without effective Machine Translation techniques, even just identifying such data in HLs is not a trivial task. Fortunately many of such comparable doc- uments are presented in multiple data modalities (text, image and video), because press releases with multimedia elements generate up to 77% more views than text-only releases (Newswire, 2011). In fact, they often contain the same or similar images and videos, which are language- independent. In this paper we propose to use images as a hub to automatically discover comparable corpora. Then we will apply Entity Discovery and Linking (EDL) techniques in HLs to extract entity knowl- edge, and project results back to LLs by leverag- ing multi-source multi-media techniques. In the following we will elaborate motivations and de- tailed methods for two most important EDL com- ponents: name tagging and Cross-lingual Entity Linking (CLEL). For CLEL we choose Chinese as the LL and English as HL because Chinese- to-English is one of the few language pairs for which we have ground-truth annotations …

Content truncated to 500 words