Using Word Embeddings to Translate Named Entities

Using Word Embeddings to Translate Named Entities Octavia-Maria S¸ulea1, 2, 3, Sergiu Nisioi1, 2, Liviu P. Dinu1, 2 Faculty of Mathematics and Computer Science, University of Bucharest1, Center for Computational Linguistics, University of Bucharest2, Bitdefender Romania 3 14 Academiei Street, Bucharest, Romania1, 7 Edgar Quinet Street, Bucharest, Romania2, 24 Delea Veche Street, Bucharest, Romania3 mary.octavia@gmail.com, sergiu.nisioi@gmail.com, liviu.p.dinu@gmail.com Abstract In this paper we investigate the usefulness of neural word embeddings in the process of translating Named Entities (NEs) from a resource-rich language to a language low on resources relevant to the task at hand, introducing a novel, yet simple way of obtaining bilingual word vectors. Inspired by observations in (Mikolov et al., 2013b), which show that training their word vector model on comparable corpora yields comparable vector space representations of those corpora, reducing the problem of translating words to finding a rotation matrix, and by results in (Zou et al., 2013), which showed that bilingual word embeddings can improve Chinese Named Entity Recognition (NER) and English to Chinese phrase translation, we use the sentence-aligned English-French EuroParl corpora and show that word embeddings extracted from a merged corpus (corpus resulted from the merger of the two aligned corpora) can be used to NE translation. We extrapolate that word embeddings trained on merged parallel corpora are useful in Named Entity Recognition and Translation tasks for resource-poor languages. Keywords: Named Entity Translation, Corpus Aquisition, Word Embeddings 1. Introduction Named Entity Recognition (NER) is a complex, Infor- mation Extraction subtask, requiring several preprocessing stages (i.e. part-of-speech tagger, tokenizer) which in turn involve dedicated tools. For resource-rich languages, such as English, NER is a highly researched area with the sate- of-the-art system achieving near-human performance: 93% F1 compared to the 97% F1 obtained by human annota- tors (Marsh and Perzanowski, 1998). For other languages having fewer language processing tools and especially task specific manually annotated data, NER is still a challenging task. Word embeddings have been recently used as features to improve existing monolingual NER systems ((Katha- rina Sienˇcnik, 2015), (Demir and Ozgur, 2014)), or to aid the translation of NEs (Zirikly, 2015). Previous to this, (Shao and Ng, 2004) reported using word embeddings as part of a larger system that extracts named entities from comparable corpora. Others have used alignment models to extract this type of information from parallel datasets (see (Moore, 2003), (Ehrmann and Turchi, 2010)). In addition to parallel or comparable datasets, metadata information, when available, can also prove useful (Ling et al., 2011) for multilingual named entity extraction. Related to multi- lingual named entities, we note the transliteration of NEs given out of context, the decision on whether to transliterate or translate also having been investigated (Mahmoud Mah- moud Azab and Oflazer, 2013). The results of the 2015 ACL shared task on transliteration of named entities1 re- vealed that further research is necessary to obtain satisfac- tory results in this direction. 1http://www.colips.org/workshop/news2015 The closest work to our own is represented by (Zou et al., 2013), which used monolingual and bilingual word em- …

Content truncated to 500 words