Building Named Entity Recognition Taggers via Parallel Corpora

Building Named Entity Recognition Taggers via Parallel Corpora Rodrigo Agerri, Yiling Chung, Itziar Aldabe, Nora Aranberri, Gorka Labaka, German Rigau IXA NLP Group, University of the Basque Country UPV/EHU rodrigo.agerri@ehu.eus,lydia193@gmail.com,{itziar.aldabe,nora.aranberri,gorka.labaka,german.rigau@ehu.eus} Abstract The lack of hand curated data is a major impediment to developing statistical semantic processors for many of the world languages. Our paper aims to bridge this gap by leveraging existing annotations and semantic processors from multiple source languages by projecting their annotations via the statistical word alignments traditionally used in Machine Translation. Taking the Named Entity Recognition (NER) task as a use case, this work presents a method to automatically induce Named Entity annotated data using parallel corpora without any manual intervention. The projected annotations can then be used to automatically generate semantic processors for the target language helping to overcome the lack of training data for a given language. The experiments are focused on 4 languages: German, English, Spanish and Italian, and our empirical evaluation results show that our method obtains competitive results when compared with models trained on gold-standard, albeit out-of-domain, data. The results point out that our projection algorithm is effective to transport NER annotations across languages thus providing a fully automatic method to obtain NER taggers for as many as the number of languages aligned in parallel corpora. Every resource generated (training data, manually annotated test set and NER models) is made publicly available for its use and to facilitate reproducibility of results. Keywords: Named Entity Recognition, Information Extraction, Multilingual Language Resources 1. Introduction The best results for every type of semantic processing task are currently obtained by supervised corpus-based ap- proaches. This means that manually annotated data is re- quired to learn probabilistic models from the data. This poses a major obstacle to developing semantic processors whenever there is not manually annotated data for a seman- tic task in a given language. In most cases, manually anno- tating text for every single specific need is generally inef- ficiently slow and, in most cases, unaffordable in terms of human resources and economic costs. Instead, we would like to be able to use already available semantic processors and texts in other languages to get a good statistical model for a new target language. Our method leverages existing semantic processors and an- notations to overcome the lack of annotation data for a given language. The intuition is to transfer or project se- mantic annotations, from multiple sources to a target lan- guage, by statistical word alignment methods applied to parallel texts (Och and Ney, 2000; Liang et al., 2006). The projected annotations could then be used to automatically generate semantic processors for the target language. In this way we would be able to provide semantic processors without training data for a given language. Thus, this means that the problem can be decomposed into two smaller inter-related ones: (i) How to project semantic annotations across languages via parallel texts with a suffi- cient acceptable quality to train semi- or weakly-supervised semantic processors and (ii) how to effectively …

Content truncated to 500 words