Expanding Chinese Sentiment Dictionaries from Large Scale Unlabeled Corpus

Expanding Chinese sentiment dictionaries from large scale unlabeled corpus Hongzhi Xu, Kai Zhao, Likun Qiu, Changjian Hu NEC Laboratories China. Tsinghua Science Park, Haidian District, Beijing 100084, China. {xu hongzhi, zhao kai, qiu likun, hu changjian}@nec.cn Abstract. Unsupervised sentiment classification usually needs a user defined sentiment dic- tionary. However, the existing dictionaries in Chinese are insufficient, for example, the inter- section rate of two popular Chinese sentiment dictionaries HowNet and NTUSD is less than 10%. In this paper, we present a method to help expand the dictionaries with more sentiment words by ranking them through link analysis based on a word graph constructed from a large unlabeled corpus. Meanwhile, our method could compute a sentiment polarity strength for each word in the new dictionaries. Manual evaluation has shown that our method has a high precision to expand the dictionaries. Experiments for sentiment classification have shown that the new dictionaries with the polarity strength for each word given by our algorithm are effective to improve the performance. As a byproduct, our algorithm could also discover the errors existing in current dictionaries. 1 Introduction Sentiment analysis becomes more and more important as various user generated content(UGC) appears on the web, such as product reviews and personal blogs. Unsupervised sentiment clas- sification has a great advantage that it does not need a large expensive labeled corpus but only a user defined sentiment dictionary. However, the current existing dictionaries(such as HowNet and NTUSD) are insufficient of their vocabularies. Many sentiment words are not included in the current Chinese sentiment dictionaries. For example, HowNet contains 3969 positive words and 3755 negative words; NUTSD contains 2648 positive words and 7742 negative words. Among them, only 669 positive words and 877 negative words are shared. Chinese idioms usually express strong sentiment, such as ‘励精图治’(to make great efforts to do sth.), ‘欺世盗名’(to win fame by cheating the world), however, they are not included in the dictionaries. Typical existing algorithms for sentiment dictionary extension use patterns to construct a graph that reflects the relation between words, clustering algorithm is then applied on this graph to get positive and negative clusters. The basic idea is that multiple sentiment words are usually used together at the same time in one sentence, and they express sentiment with same polarity. The intuition is that, if a word mostly occurred with positive/negative sentiment words based on the current dictionaries, then we can guess it is also a positive/negative sentiment word. For example, ‘挺拔’(tall and straight) is usually co-occurred with positive words: ‘高大’(tall), ‘气 派’(style), ‘刚强’(strong), ‘秀丽’(elegant), but rarely co-occurred with negative words, so we can guess that ‘挺拔’is also a positive word. In this work, we develop a new technique to deal with this problem. Similarly, by observing a large amount of sentiment word co-occurrences, we can construct a graph based on them, within which the polarity of some words that are defined in current sentiment dictionaries are known. So, our method is also based on a graph. However, we use syntactic analysis technology …

Content truncated to 500 words