Efficient Support Vector Classifiers for Named Entity Recognition

Efficient Support Vector Classifiers for Named Entity Recognition Hideki Isozaki and Hideto Kazawa NTT Communication Science Laboratories Nippon Telegraph and Telephone Corporation 2-4 Hikari-dai, Seika-cho, Soraku-gun, Kyoto, 619-0237, Japan � isozaki,kazawa � @cslab.kecl.ntt.co.jp Abstract Named Entity (NE) recognition is a task in which proper nouns and numerical information are ex- tracted from documents and are classified into cat- egories such as person, organization, and date. It is a key technology of Information Extraction and Open-Domain Question Answering. First, we show that an NE recognizer based on Support Vector Ma- chines (SVMs) gives better scores than conventional systems. However, off-the-shelf SVM classifiers are too inefficient for this task. Therefore, we present a method that makes the system substantially faster. This approach can also be applied to other simi- lar tasks such as chunking and part-of-speech tag- ging. We also present an SVM-based feature selec- tion method and an efficient training method. 1 Introduction Named Entity (NE) recognition is a task in which proper nouns and numerical information in a docu- ment are detected and classified into categories such as person, organization, and date. It is a key technol- ogy of Information Extraction and Open-Domain Question Answering (Voorhees and Harman, 2000). We are building a trainable Open-Domain Question Answering System called SAIQA-II. In this paper, we show that an NE recognizer based on Support Vector Machines (SVMs) gives better scores than conventional systems. SVMs have given high per- formance in various classification tasks (Joachims, 1998; Kudo and Matsumoto, 2001). However, it turned out that off-the-shelf SVM classifiers are too inefficient for NE recognition. The recognizer runs at a rate of only 85 bytes/sec on an Athlon 1.3 GHz Linux PC, while rule-based systems (e.g., Isozaki, (2001)) can process several kilobytes in a second. The major reason is the inefficiency of SVM classifiers. There are other reports on the slowness of SVM classifiers. An- other SVM-based NE recognizer (Yamada and Mat- sumoto, 2001) is 0.8 sentences/sec on a Pentium III 933 MHz PC. An SVM-based part-of-speech (POS) tagger (Nakagawa et al., 2001) is 20 tokens/sec on an Alpha 21164A 500 MHz processor. It is difficult to use such slow systems in practical applications. In this paper, we present a method that makes the NE system substantially faster. This method can also be applied to other tasks in natural language processing such as chunking and POS tagging. An- other problem with SVMs is its incomprehensibil- ity. It is not clear which features are important or how they work. The above method is also useful for finding useless features. We also mention a method to reduce training time. 1.1 Support Vector Machines Suppose we have a set of training data for a two- class problem: ���� � � �������� � , where � �  is a feature vector of the -th sample in the training data and �   "!$#%��&#"' is the label for the sample. The goal is to find a decision func- tion that accurately predicts � for unseen � . …

Content truncated to 500 words