Maximum Entropy Models for Named Entity Recognition

Maximum Entropy Models for Named Entity Recognition Oliver Bender � and Franz Josef Och � and Hermann Ney � � Lehrstuhl f¨ur Informatik VI � Information Sciences Institute Computer Science Department University of Southern California RWTH Aachen - University of Technology Marina del Rey, CA 90292 D-52056 Aachen, Germany och@isi.edu � bender,ney � @cs.rwth-aachen.de Abstract In this paper, we describe a system that applies maximum entropy (ME) models to the task of named entity recognition (NER). Starting with an annotated corpus and a set of features which are easily obtainable for almost any language, we first build a baseline NE recognizer which is then used to extract the named entities and their context information from additional non- annotated data. In turn, these lists are incor- porated into the final recognizer to further im- prove the recognition accuracy. 1 Introduction In this paper, we present an approach for extracting the named entities (NE) of natural language inputs which uses the maximum entropy (ME) framework (Berger et al., 1996). The objective can be described as follows. Given a natural input sequence ��� � �  �  � � we choose the NE tag sequence  � � � �      � with the highest probability among all possible tag sequences:   � � �    ! " #%$'&)(  � �+* � � �',.- The argmax operation denotes the search problem, i.e. the generation of the sequence of named entities. Ac- cording to the CoNLL-2003 competition, we concentrate on four types of named entities: persons (PER), locations (LOC), organizations (ORG), and names of miscellaneous entities (MISC) that do not belong to the previous three groups, e.g. [PER Clinton] ’s [ORG Ballybunion] fans in- vited to [LOC Chicago] . Additionally, the task requires the processing of two different languages from which only English was spec- ified before the submission deadline. Therefore, the system described avoids relying on language-dependent knowledge but instead uses a set of features which are easily obtainable for almost any language. The remainder of the paper is organized as follows: in section 2, we outline the ME framework and specify the features that were used for the experiments. We describe the training and search procedure of our approach. Sec- tion 3 presents experimental details and shows results ob- tained on the English and German test sets. Finally, sec- tion 4 closes with a summary and an outlook for future work. 2 Maximum Entropy Models For our approach, we directly factorize the posterior probability and determine the corresponding NE tag for each word of an input sequence. We assume that the decisions only depend on a limited window of �  /.0 2130 �4�52160  �5 /70 around the current word �8 and on the two predecessor tags. Thus, we obtain the follow- ing second-order model: $'&)(  � � * � � � , � � 9 : � $'&)( ;*  )1 � � < � � � , m� =?>A@!B …

Content truncated to 500 words