Adaptive Word Sense Tagging on Chinese Corpus

Adaptive Word Sense Tagging on Chinese Corpus Sue-Jin Ker Department of Computer& Information Science Soochow University Taipei, Taiwan ksj@cis.scu.edu.tw Jen-Nan Chen Department of Computer Science & Information Engineering Ming Chuan University Taipei, Taiwan jnchen@mcu.edu.tw Abstract This study describes a general framework for adaptive word sense disambiguation. The proposed framework begins with knowledge acquisition from the relatively easy context of a corpus. The proposed framework heavily relies on the adaptive step that enriches the initial knowledge base with knowledge gleaned from the partially disambiguated text. Once adjusted to fit the text at hand, the knowledge base is applied to the text again to finalize the disambiguation decision. The effectiveness of this approach was examined through sentences from the Sinica corpus. Experimental results indicated that adaptation significantly improved the performance of WSD. Moreover, the adaptive approach, achieved an applicability improvement from 33.0% up to 74.9% with a comparable precision. 1 Introduction Word sense disambiguation is a long-standing problem in natural language understanding. Statistically acquiring sufficient knowledge about a language to build a robust WSD system is extremely difficult. For such a system to be efficient, a large mass of balanced materials must be gathered to cover many idiosyncratic facets of the language. Three issues must be addressed in a lexicalized statistical word sense disambiguation (WSD) model: data sparseness, lack of abstraction, and static learning. First, a word-based model has a multiplicity of parameters that are difficult to measure consistently, even with an extremely large corpus. Under-trained models lead to low precision. Second, word-based models lack a crucial degree of abstraction for a broad coverage system. Third, a static WSD model is probably neither robust nor portable, since it is difficult to construct a model relevant to a broad range of unrestricted texts. Several WSD systems have been created that apply word-based models to a specific domain to disambiguate senses appearing in generally easy contexts with a large number of typically salient words. In an unrestricted text, however, the context is usually diverse and difficult to capture with a lexicalized model; therefore, a corpus-trained system is unlikely to transfer suitably to a new domain. Generality and adaptability are, therefore, essential to a robust and portable WSD system. An adaptive system, armed with an initial knowledge base extracted from defined words, is superior in two ways to static word-based models trained on a corpus. First, the initial knowledge is sufficiently rich and unbiased for a large portion of text to be disambiguated correctly. Second, based on the initial disambiguation, an adaptation step can then be implemented render the knowledge base more relevant to the task, thus resulting in broader and more precise WSD. This study explores in detail whether word-based knowledge provides a general solution for disambiguating contexts of unrestricted texts. This method assumes that a major part of a given text is easy or prototypical and, therefore, understandable using general knowledge. Adapting contextual representation of word senses to those in the easy context, will hopefully allow us to PACLIC 18, December 8th-10th, 2004, …

Content truncated to 500 words