Identification and Classification of Proper Nouns in Chinese Texts

Identification and Classification of Proper Nouns in Chinese Texts Hsin-Hsi Chen and Jen-Chang Lee Department of Computer Science and Information Engineering National Taiwan University Taipei, TAIWAN, R.O.C. hh chen@csie.ntu, edu.tw Abstract Various strategies are proposed to identify and classify three types of proper nouns in Chinese texts. Clues from character, sentence and paragraph levels are employed to resolve Chinese personal names. Character, Syllable and Frequency Conditions are presented to treat transliterated personal names, To deal with organization names, keywords, prefix, word association and parts-of-speech are applied. For fair evaluation, large scale test data are selected from six sections of a newspaper. The precision and the recall for these three types are (88.04%, 92.56%), (50.62%, 71.93%) and (61.79%, 54.50%), respectively. When the former two types are regarded as a category, the performance becomes (81.46%, 91.22%). Compared with other approaches, our approach has better performance and our classification is automatic. 1. Introduction A Chinese sentence is composed of a string of characters without any word boundaries, so that to segment Chinese sentences is indispensable in Chinese language processing (Chen, 1990; Chen, 1994). Many word segmentation techniques (Chen & Liu, 1992; Chiang et al., 1992; Sproat & Shih, 1990) have been developed. However, the resolution of unknown words, i.e., those words not in the dictionaries, form the bottleneck. Some papers (Fung & Wu, 1994; Wang et al., 1994) based on Smadja's paradigm (1993) learned an aided dictionary from a corpus to reduce the possibility of unknown words. Chang et al. (1992) proposed a method to extract Chinese personal names from an 11,000-word corpus, and reported 91.87% precision and 80.67% recall. Wang et al. (1992) recognized unregistered names on the basis of titles and a surname-driven rule. Linet al. (1993) presented a model to tackle a very restrictive form of unknown words. Sproat et al. (1994) considered Chinese personal names and transliterations of foreign words. Their performance was 61.83% precision and 80.99% recall on an 12,000-Chinese-character corpus. This paper deals with three kinds of proper nouns - say, Chinese personal names, transliterated personal names and organization names. We not only tell if an unknown word is a proper noun, but also assign it a suitable semantic feature. In other words, '~?~4~ ~' (George Bush) will have a feature of male transliterated personal name when it is identified. Such a rigid treatment will be helpful for further applications such as anaphora resolution (Chen, 1992), sentence alignment (Chert & Chert, 1994; Chen& Wu, 1995), etc. Section 2 describes the training corpora and the testing corpus we used. Sections 3, 4 and 5 propose tile identification and classification methods of Chinese personal names, transliterated personal names and organization names, respectively. Section 6 presents two applications. Section 7 concludes the remarks. 2. Training Corpora and Testing Corpus The proposed methods in this paper integrate the rule-based and the statistics-based models, so that training corpora are needed. To test the performance of language models, a good testing corpus is also necessary. This section introduces all the corpora that are …

Content truncated to 500 words