Similarity Based Chinese Synonym Collocation Extraction

Computational Linguistics and Chinese Language Processing Vol. 10, No. 1, March 2005, pp.123-144 123 © The Association for Computational Linguistics and Chinese Language Processing Similarity Based Chinese Synonym Collocation Extraction Wanyin Li∗, Qin Lu∗ and Ruifeng Xu∗ Abstract Collocation extraction systems based on pure statistical methods suffer from two major problems. The first problem is their relatively low precision and recall rates. The second problem is their difficulty in dealing with sparse collocations. In order to improve performance, both statistical and lexicographic approaches should be considered. This paper presents a new method to extract synonymous collocations using semantic information. The semantic information is obtained by calculating similarities from HowNet. We have successfully extracted synonymous collocations which normally cannot be extracted using lexical statistics. Our evaluation conducted on a 60MB tagged corpus shows that we can extract synonymous collocations that occur with very low frequency and that the improvement in the recall rate is close to 100%. In addition, compared with a collocation extraction system based on the Xtract system for English, our algorithm can improve the precision rate by about 44%. Keywords: Lexical Statistics, Synonymous Collocations, Similarity, Semantic Information 1. Introduction A collocation refers to the conventional use of two or more adjacent or distant words which hold syntactic and semantic relations. For example, the conventional expressions “warm greetings”, “broad daylight”, “思想包袱”, and “托运行李” all are collocations. Collocations bear certain properties that have been used to develop feasible methods to extract them automatically from running text. Since collocations are commonly found, they must be recurrent. Therefore, their appearance in running text should be statistically significant, making it feasible to extract them using the statistical approach. ∗ Department of Computing, The Hong Kong Polytechnic University, Hung Hom, Kowloon, Hong Kong Tel: +852-27667326; +852-27667247 Fax:+852-27740842 E-mail: {cswyli, csluqin, csrfxu}@comp.polyu.edu.hk 124 Wanyin Li et al. A collocation extraction system normally starts with a so-called headword (sometimes also called a keyword) and proceeds to find co-occurring words called the collocated words. For example, given the headword “基本”, such bi-gram collocations as “基本理论”, “基本工 作”, and, “基本原因” can be found using an extraction system where “理论”, “工作”, and “原 因” are called collocated words with respect to the headword “基本.” Many collocation extraction algorithms and systems are based on lexical statistics [Church and Hanks 1990; Smadja 1993; Choueka 1993; Lin 1998]. As the lexical statistical approach was developed based on the recurrence property of collocations, only collocations with reasonably good recurrence can be extracted. Collocations with low occurrence frequency cannot be extracted, thus affecting both the recall rate and precision rate. The precision rate achieved using the lexical statistics approach can reach around 60% if both word bi-gram extraction and n-gram extraction are employed [Smadja 1993; Lin 1997; Lu et al. 2003]. The low precision rate is mainly due to the low precision rate of word bi-gram extraction as only about a 30% - 40% precision rate can be achieved for word bi-grams. The semantic information is largely ignored by statistics- based collocation extraction systems …

Content truncated to 500 words