HowtogetaChineseName(Entity): Segmentation and Combination Issues

HowtogetaChineseName(Entity): Segmentation and Combination Issues Hongyan Jing Radu Florian Xiaoqiang Luo Tong Zhang Abraham Ittycheriah IBM T.J. Watson Research Center Yorktown Heights, NY 10598 � hjing,raduf,xiaoluo,tzhang,abei � @us.ibm.com Abstract When building a Chinese named entity recognition system, one must deal with certain language-specific issues such as whether the model should be based on characters or words. While there is no unique answer to this question, we discuss in detail advantages and disadvantages of each model, identify problems in segmen- tation and suggest possible solutions, pre- senting our observations, analysis, and experimental results. The second topic of this paper is classifier combination. We present and describe four classifiers for Chinese named entity recognition and describe various methods for combining their outputs. The results demonstrate that classifier combination is an effective tech- nique of improving system performance: experiments over a large annotated corpus of fine-grained entity types exhibit a 10% relative reduction in F-measure error. 1 Introduction Named entity (NE) recognition has drawn much at- tention in recent years. It was a designated task in a number of conferences, including the Mes- sage Understanding Conferences (MUC-6, 1995; MUC-7, 1997), the Information Retrieval and Ex- traction Conference (IREX, 1999), the Conferences on Natural Language Learning (Tjong Kim Sang, 2002; Tjong Kim Sang and De Meulder, 2003), and the recent Automatic Content Extraction Con- ference (ACE, 2002). A variety of algorithms have been proposed for NE recognition. Many of these algorithms are, in principle, language-independent. However, when applying these algorithms to languages such as Chinese and Japanese, we must deal with cer- tain language-specific issues: for example, should we build a character-based model or a word-based model? how do word segmentation errors affect NE recognition? how should word segmentation and NE recognition interact with each other? Besides word segmentation related issues, Chinese does not have capitalization, which is a very useful feature in iden- tifying NEs in languages such as English, Spanish, or Dutch. How does the lack of features such as cap- italization affect the performance? In the first part of this paper, we discuss these language-specific issues in Chinese NE recogni- tion. In particular, we use a hidden Markov model (HMM) system as an example, and discuss various issues related to applying the HMM classifier to Chi- nese. The HMM classifier is similar to the one de- scribed in (Bikel et al., 1999). In the second part of this paper, we investigate the combination of a set of diverse NE recognition classifiers. Four statistical classifiers are combined in the experiments, including the above-mentioned hidden Markov model classifier, a transformation- based learning classifier (Brill, 1995; Florian and Ngai, 2001), a maximum entropy classifier (Ratna- parkhi, 1999), and a robust risk minimization classi- fier (Zhang et al., 2002). The remainder of this paper is organized as fol- lows: Section 2 describes the experiment data, Sec- tion 3 discusses specific issues related to Chinese NE recognition, Section 4 presents the four classi- fiers and approaches to combining these classifiers. 2 Data We used three annotated …

Content truncated to 500 words