Design Challenges and Misconceptions in Named Entity Recognition

Proceedings of the Thirteenth Conference on Computational Natural Language Learning (CoNLL), pages 147–155, Boulder, Colorado, June 2009. c⃝2009 Association for Computational Linguistics Design Challenges and Misconceptions in Named Entity Recognition∗† ‡ Lev Ratinov Dan Roth Computer Science Department University of Illinois Urbana, IL 61801 USA {ratinov2,danr}@uiuc.edu Abstract We analyze some of the fundamental design challenges and misconceptions that underlie the development of an efficient and robust NER system. In particular, we address issues such as the representation of text chunks, the inference approach needed to combine local NER decisions, the sources of prior knowl- edge and how to use them within an NER system. In the process of comparing several solutions to these challenges we reach some surprising conclusions, as well as develop an NER system that achieves 90.8 F1 score on the CoNLL-2003 NER shared task, the best reported result for this dataset. 1 Introduction Natural Language Processing applications are char- acterized by making complex interdependent deci- sions that require large amounts of prior knowledge. In this paper we investigate one such application– Named Entity Recognition (NER). Figure 1 illus- trates the necessity of using prior knowledge and non-local decisions in NER. In the absence of mixed case information it is difficult to understand that ∗ The system and the Webpages dataset are available at: http://l2r.cs.uiuc.edu/∼cogcomp/software.php † This work was supported by NSF grant NSF SoD-HCER- 0613885, by MIAS, a DHS-IDS Center for Multimodal In- formation Access and Synthesis at UIUC and by an NDIIPP project from the National Library of Congress. ‡ We thank Nicholas Rizzolo for the baseline LBJ NER system, Xavier Carreras for suggesting the word class models, and multiple reviewers for insightful comments. SOCCER - [PER BLINKER] BAN LIFTED . [LOC LONDON] 1996-12-06 [MISC Dutch] forward [PER Reggie Blinker] had his indefinite suspension lifted by [ORG FIFA] on Friday and was set to make his [ORG Sheffield Wednesday] comeback against [ORG Liverpool] on Saturday . [PER Blinker] missed his club’s last two games after [ORG FIFA] slapped a worldwide ban on him for appearing to sign contracts for both [ORG Wednesday] and [ORG Udinese] while he was playing for [ORG Feyenoord]. Figure 1: Example illustrating challenges in NER. “BLINKER” is a person. Likewise, it is not obvi- ous that the last mention of “Wednesday” is an orga- nization (in fact, the first mention of “Wednesday” can also be understood as a “comeback” which hap- pens on Wednesday). An NER system could take ad- vantage of the fact that “blinker” is also mentioned later in the text as the easily identifiable “Reggie Blinker”. It is also useful to know that Udinese is a soccer club (an entry about this club appears in Wikipedia), and the expression “both Wednesday and Udinese” implies that “Wednesday” and “Udi- nese” should be assigned the same label. The above discussion focuses on the need for ex- ternal knowledge resources (for example, that Udi- nese can be a soccer club) and the need for non- local features to leverage the multiple occurrences …

Content truncated to 500 words