An Agent-based Approach to Chinese Named Entity Recognition

An Agent-based Approach to Chinese Named Entity Recognition Shiren Ye Tat-Seng Chua Liu Jimin School of Computing, National University of Singapore, Singapore, 117543 yesr@comp.nus.edu.sg chuats@comp.nus.edu.sg Liujm@comp.nus.edu.sg Abstract Chinese NE (Named Entity) recognition is a difficult problem because of the uncertainty in word segmentation and flexibility in language structure. This paper proposes the use of a rationality model in a multi-agent framework to tackle this problem. We employ a greedy strategy and use the NE rationality model to evaluate and detect all possible NEs in the text. We then treat the process of selecting the best possible NEs as a multi-agent negotiation problem. The resulting system is robust and is able to handle different types of NE effectively. Our test on the MET-2 test corpus indicates that our system is able to achieve high F1 values of above 92% on all NE types. 1. Introduction Named entity (NE) recognition is a fundamental step to many language processing tasks. It was a basic task of the Message Understanding Conference (MUC) and has been studied intensively. Palma & Day (97) reported that person (PER), location (LOC) and organization (ORG) names are the most difficult sub-tasks as compared to other entities as defined in MUC. This paper thus focuses only on the recognition of PER, LOC and ORG entities. Recent research on NE recognition has been focused on the machine learning approach, such as the transformation-based learning (Aberdeen 95), hidden Markov model (Bikel et al. 97), decision tree (Sekin et al. 98), collocation statistics (Lin 98), maximum entropy model (Borthwick 99), and EM bootstrapping (Cucerzan & Yarowsky 99). Other than English, several recent works examined the extraction of information from Spanish, Chinese, and Japanese (Isozaki 01). Most approaches for Chinese NE recognition used handcrafted rules, supplemented by word or character frequency statistics. These methods require a lot of resources to model the NEs. Chen et al. (98) used 1-billion person name dictionary and employed mainly internal word statistics with no generalization. Yu et al. (98) employed a common framework to model both the context and information residing within the entities, and performed rule generalization using POS (part-of-speech) and some semantic tags. A similar system is also reported in Luo & Song (01). Chinese NE recognition is much more difficult than that in English due to two major problems. The first is the word segmentation problem (Sproat et al. 96, Palmer 97). In Chinese, there is no white space to delimit the words, where a word is defined as consisting of one or more characters representing a linguistic token. Word is a vague concept in Chinese, and Palmer (97) showed that even native speakers could only achieve about 75% agreement on “correct” segmentation. As word segmentation is the basic initial step to almost all linguistic analysis tasks, many techniques developed in English NLP cannot be applied to Chinese. Second, there is no exterior feature (such as the capitalization) to help identify the NEs, which share many common characters with non-NE (or common words). For example, …

Content truncated to 500 words