The Multilingual Named Entity Recognition Framework

The Multilingual Named Entity Recognition Framework Thierry Poibeau and the INaLCO Named Entity Group' INaLCO/CRIIVI 2 rue de Lille 75007 Paris Abstract This paper presents a multilingual system designed to recognize named entities in a wide variety of languages (currently more than 12 languages are concerned). The system includes original strategies to deal with a wide variety of encoding character sets, analysis strategies and algorithms to process these languages. 1 Introduction Since the MUC conferences about Information Extraction, named entity recognition (NERC) is a well-established task in the NLP community (MUC-6, 1995). Examples of named entities are person names, location and company names, date and time indications, etc. A lot of systems have been developed to perform this task, ranging from manually created rule-based systems to fully automatic learning-based systems. We will shortly present these technologies below.' Even if a lot of systems have been developed for languages such as English or Japanese, a large range of languages do not have access to such a technology. We propose an open framework to develop resources and tools for named entity recognition. A team of computational linguist students develops this The members of the INaLCO Named Entity Group are: A. Acoulon, C. Avaux, L. Beroff-Beneat-, A. Cadeau, M. Calberg, A. Delale, L. De Temmerman, A.-L. Guenet, D. Huis, M. Jamalpour, A. Krul, A. Marcus, F. Picoli and C. Plancq. projecti, so that it also has pedagogic purposes. But, even so, the project seems to be sufficiently attractive to interest industrial partners. We describe the different approaches for named entity recognition. We then present the project and the different analysis techniques used. We will conclude with some considerations on evaluation and future work. 2 State of the art NERC systems In this section, we examine the different approaches to named entity recognition. We then examine previous experiments to compare systems and techniques. Sekine and Eriguchi (2000) present an interesting classification of •named entity recognition systems. Manually created rule-based systems. In this kind of system, developers initially elaborate a set of patterns that will be applied on the text to accurately recognize and tag named entities. Nearly all classical MUC systems were using this approach until the mid' 90s, and most of them are still using this kind of technique •(MUC-6, 1995). Fully automatic learning-based systems. These systems are using Machine Learning (ML) techniques to learn a model in order to accurately tag the texts. The result of the learning task can be a set of rules, a decision tree or a set of numeric data. Note that a human cannot always revise the result if the learning algorithm used does not provide a readable output. These systems are now very popular in the IE community (Bikel et al., 1997) (Collins and Singer, 1999), even if they were initially rather dedicated to audio corpora. 155 Text Rule-based Systems ••■■•• dictionary gralTITTIat 1/ (4) Revision mechanisms (1) Lexical analsis (2) Grammar application (3) Dynamic acquisition from the text A Annotated Text if • Mixed approach. …

Content truncated to 500 words