The Improving Techniques for Disambiguating Non-alphabet Sense Categories

The Improving Techniques for Disambiguating Non-alphabet Sense Categories Feng-Long Hwang, Ming-Shing Yu, Min-Jer Wu Department of Applied Mathematics, National Chung-Hsing University Page 67 ~ 86 Proceedings of Research on Computational Linguistics Conference XIII (ROCLING XIII) Taipei, Taiwan 2000-08-24/2000-08-25 67 The Improving Techniques for Disambiguating Non-alphabet Sense Categories Feng-Long Hwangψ, Ming-Shing Yu, Min-Jer Wu Department of Applied Mathematics, National Chung-Hsing University, Taichung 40227, Taiwan, flhwang@mail.lctc.edu.tw, msyu@dragon.nchu.edu.tw ABSTRACT Usually, there are various non-alphabet symbols (“/”, “:”, “-”, etc.) occurring in Mandarin texts. Such symbols may be pronounced more than one oral expression with respect to its sense category. In our previous works, we proposed the multi-layer decision classifier to disambiguate the sense category of non-alphabet symbols; the elementary feature is the statistical probability of token adopting the Bayesian rule. This paper adopts more features of tokens in sentences. Three techniques are further proposed to improve the performance. Experiments show that the proposed techniques can disambiguate the sense category of target symbols quite well, even with small size of data. The precision rates for inside and outside tests are upgraded to 99.6% and 96.5% by using more features of token and techniques. Key Words: Multi-layer decision classifier, Bayesian rule, word sense disambiguation, voting scheme, pattern table. 1. Introduction Various homographs or non-alphabet symbols in the Mandarin (but not limited to) occur frequently. The patterns containing these symbols may be pronounced with respect to its semantic sense. The non-alphabet symbols are defined: the symbols which are not the Mandarin characters (字) and may be pronounced different oral expressions. We call such phenomenon oral ambiguity. The purpose of word sense disambiguation (WSD) is to identify the most possible category among candidate’s sense category. It is important to disambiguate the word sense automatically for the natural language processing (NLP). Many works [Brown etc., 1991], [Fujii and Inue,1998] and [Ide and Veronis,1998], addressed WSD problems in the past. In our previous works [Hwang, etc., 1999a; Hwang, etc., 1999b], we proposed the ψ Correspondence author. 68 multi-layer decision classifier (MLDC) to predict the sense category, in which the voting scheme is used to predict the final category. Even though the domains of sense in the paper just focus on three non-alphabet symbols, the proposed approach can be extended into other symbols in Mandarin and related ambiguity problems. The features of token and improving techniques described in this paper will be employed in the 2nd layer classifier. The main domain will focus on the improvements for the 2nd layer decision classifier. The model of our previous works is regarded as the baseline system. Comparing with the baseline model, the proposed features of token and techniques in this paper improve the performance of inside test from 97.8 to 99.6% and outside test from 93.0 to 96.6%. The paper is organized as follows: related information and previous works will be described first. Section 3 elaborates the principal techniques for 2nd layer classifier in MLDC. Section 4 focuses on the evaluation for empirical features. Some improving techniques are proposed in section 5. …

Content truncated to 500 words