Building a Multilingual Named Entity-Annotated Corpus Using Annotation Projection

Proceedings of Recent Advances in Natural Language Processing, pages 118–124, Hissar, Bulgaria, 12-14 September 2011. Building a Multilingual Named Entity-Annotated Corpus Using Annotation Projection Maud Ehrmann, Marco Turchi, Ralf Steinberger European Commission - Joint Research Centre (JRC), IPSC - GlobeSec Via Fermi 2749, 21020 Ispra (VA) - Italy name.surname@jrc.ec.europa.eu Abstract As developers of a highly multilingual named entity recognition (NER) system, we face an evaluation resource bottleneck problem: we need evaluation data in many languages, the annotation should not be too time-consuming, and the evaluation re- sults across languages should be compa- rable. We solve the problem by automat- ically annotating the English version of a multi-parallel corpus and by projecting the annotations into all the other language versions. For the translation of English entities, we use a phrase-based statistical machine translation system as well as a lookup of known names from a multilin- gual name database. For the projection, we incrementally apply different methods: perfect string matching, perfect consonant signature matching and edit distance simi- larity. The resulting annotated parallel cor- pus will be made available for reuse. 1 Introduction Named Entity recognition is a well-established task, acknowledged as fundamental to a wide va- riety of natural language processing (NLP) appli- cations (Nadeau and Sekine, 2007). As for other text mining applications, annotated corpora con- stitute a crucial and constant need for named en- tity recognition (NER). Within a development or training framework, annotated corpora are used as models from which machine learning systems, or computational linguists, can infer rules and deci- sion criteria; within an evaluation framework, they are used as a gold standard to assess systems’ per- formances and help to guide their quality improve- ment, e.g. via non-regression tests. During the last decade, several named entity (NE) annotated corpora were built, thanks to a large series of evaluation campaigns (Fort et al., 2009). However, most of these gold-standard data are available only for English or for a few lan- guages. Even if unsupervised methods tried to overcome this difficulty, the shortage of annotated data for the large majority of the world’s languages remains a problem. An obvious solution is to man- ually produce annotated corpora, but it is a com- plex and time-consuming task and it may be diffi- cult to find experts in each specific language. Beyond the scarcity of annotated corpora, an- other issue lies in the fact that annotation schemas or guidelines usually differ from one annotated corpus to another: named entity extents can be dif- ferent (e.g. inclusion or not of the function in a per- son name), as well as entity types and granularity (e.g. some corpora may consider product names, whereas others will differentiate, within this cat- egory, vehicles, awards and documents, and oth- ers will not even consider product names). Such divergences should be expected, as annotated cor- pora are built according to different applications. However, they constitute a real issue, particularly when developing or evaluating multilingual NE recognition systems and the effort to reuse exist- ing annotated data …

Content truncated to 500 words