A Three-Phase System for Chinese Named Entity Recognition

A Three-Phase System for Chinese Named Entity Recognition Conrad Chen Department of Computer Science and Information Engineering, National Chiao Tung University, Hsinchu drchen@csie.nctu.edu.tw Hsi-Jian Lee Department of Medical Informatics, Tzu Chi University, Hualien hjlee@mail.tcu.edu.tw Abstract. The handling of out-of-vocabulary (OOV) words is one of the key points to a high performance lexical analysis in natural language processing. Among all OOV words, named entities (NE) are the most productive ones. They generally constitute the most meaningful parts of sentences (persons, affairs, time, places, and objects). In this paper, we propose a three-phase “generation, filtering, and recovery” system to address the NER problem. A set of stochastic models is first used to generate all possible NE candidates. Then we treat candidate filtering as an ambiguity resolution problem. To resolve ambiguities, we adopt a maximal-matching-rule-driven lexical analyzer. Last, a pattern matching method is applied to detect and recover abnormalities in the results of the previous two phases. Pure lexical information is exploited in our system. We get a high recall of 96% with personal names (PER), satisfiable recall of 88%, 89%, and 80% with transliteration names (TRA), location names (LOC), and organization names (ORG), respectively. The overall precision and excluding rate is over 90% and 99%. 1. Introduction Words are generally the basic unit to process natural languages. However, in Chinese, sentences are composed of string of characters without any delimiters to mark word boundaries. To process Chinese, sentences must be segmented into word sequences first. Most Chinese language processing systems rely on lexicons to recognize words in sentences. Because the number of Chinese words is tremendous, it is impossible to compile all words in a lexicon. Therefore, word segmentation processes often encounters the problem of out-of-vocabulary (OOV) words. Among all OOV words, named entities are one of the most important sorts. It is impossible to list them exhaustively in a lexicon. They are the most productive type of words. Nearly no simple or unified generation rules for them exist. Besides, they are usually keywords in documents. Named entity recognition (NER) thus becomes a major task to many natural language applications, such as natural language understanding, question answering, and information retrieval. Many researches have addressed the NE recognition problem in Chinese since 1990. Most of them focused on some specific types as personal names �[5]�[13], location names �[9], organization names �[10], and transliteration names �[11]. There are also type-independent approaches of NER. However, most of these approaches need type-dependent data such as role tags. Type-independent approaches can be roughly divided into two major sorts: over-generating & disambiguating �[3]�[12] and over-segmenting & generating �[4]�[8]. Generally speaking, there are two main approaches of the above studies, rule-based models and machine learning methods. Rule-based approaches could effectively exploit human knowledge and can be tuned conveniently. On the other hand, machine learning approaches, such as maximum entropy or support vector machine, is more independent from languages and simple to implement. Rule-based approaches is slightly outperform machine learning ones in MUC-7 tests �[2]. In our consideration, rule-based approaches …

Content truncated to 500 words