JRC-NAMES: A Freely Available, Highly Multilingual Named Entity Resource

Proceedings of Recent Advances in Natural Language Processing, pages 104–110, Hissar, Bulgaria, 12-14 September 2011. JRC-NAMES: A Freely Available, Highly Multilingual Named Entity Resource Ralf Steinberger, Bruno Pouliquen, Mijail Kabadjov, Jenya Belyaeva & Erik van der Goot European Commission – Joint Research Centre Via Enrico Fermi 2749, 21027 Ispra (VA), Italy {Firstname.Lastname}@jrc.ec.europa.eu Abstract This paper describes a new, freely available, highly multilingual named entity resource for person and organisation names that has been compiled over seven years of large-scale multilingual news analysis combined with Wikipedia mining, resulting in 205,000 per- son and organisation names plus about the same number of spelling variants written in over 20 different scripts and in many more languages. This resource, produced as part of the Europe Media Monitor activity (EMM, http://emm.newsbrief.eu/overview.html), can be used for a number of purposes. These in- clude improving name search in databases or on the internet, seeding machine learning sys- tems to learn named entity recognition rules, improve machine translation results, and more. We describe here how this resource was created; we give statistics on its current size; we address the issue of morphological inflection; and we give details regarding its functionality. Updates to this resource will be made available daily. 1 Introduction The release consists of named entity lists and Java-implemented software. The software per- forms two main functionalities: (1) It recognises known names in text of any language and returns the name as it was found, its position and length, the standard variant of the name and the unique numerical name identifier. (2) It allows exporting all known name variants so that users can exploit the resource in further ways. JRC-Names is the result of a multi-year large- scale effort to recognise new person and organi- sation names in up to 100,000 news articles per day in up to 20 different languages1, to automati- 1 EMM-NewsExplorer (http://emm.newsexplorer.eu/) currently extracts new names from news articles in cally recognise which newly found names are variants of each other, to enhance the name list with additional information extracted from Wikipedia, and to manually improve the data- base entries for the most frequently found names. While the Named Entity Recognition (NER) module itself is not part of the release, updates to JRC-Names will be made available daily so that users will always have access to the latest named entity (NE) list.2 In the following sections, we describe possible uses of this resource ( 2), list other available NE resources ( 3), explain very briefly how the re- source was built ( 4), give some statistics on the resource and provide technical details on the re- leased software ( 5). 2 Uses of this named entity resource The tool serves many purposes and addresses various problems, including the following: (a) Proper names are a problem when searching databases, the internet and other repositories, because variants of searched names are often not found (Stern & Sagot 2010). This results in non-optimal use and exploitation of re- positories for documents, images and audio- visual content. …

Content truncated to 500 words