Is Twitter A Better Corpus for Measuring Sentiment Similarity?

Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 897–902, Seattle, Washington, USA, 18-21 October 2013. c⃝2013 Association for Computational Linguistics Is Twitter A Better Corpus for Measuring Sentiment Similarity? Shi Feng1, Le Zhang1, Binyang Li2,3, Daling Wang1, Ge Yu1, Kam-Fai Wong3 1Northeastern University, Shenyang, China 2University of International Relations, Beijing, China 3The Chinese University of Hong Kong, Shatin, N.T., Hong Kong {fengshi,wangdaling,yuge}@ise.neu.edu.cn, zhang777le@gmail.com {byli,kfwong}@se.cuhk.edu.hk Abstract Extensive experiments have validated the ef- fectiveness of the corpus-based method for classifying the word’s sentiment polarity. However, no work is done for comparing d- ifferent corpora in the polarity classification task. Nowadays, Twitter has aggregated huge amount of data that are full of people’s senti- ments. In this paper, we empirically evaluate the performance of different corpora in sen- timent similarity measurement, which is the fundamental task for word polarity classifica- tion. Experiment results show that the Twitter data can achieve a much better performance than the Google, Web1T and Wikipedia based methods. 1 Introduction Measuring semantic similarity for words and short texts has long been a fundamental problem for many applications such as word sense disambiguation, query expansion, search advertising and so on. Determining the word’s polarity plays a critical role in opinion mining and sentiment analysis task. Usually we can detect the word’s polarity by mea- suring it’s semantic similarity with a positive seed word sep and a negative seed word sen respectively, as shown in Formula (1): SO(w) = sim(w, sep) − sim(w, sen) (1) where sim(wi, wj) is the semantic similarity mea- surement method for the given word wi and wj. A lot of papers have been published for designing ap- propriate similarity measurements. One direction is to learn similarity from the knowledge base or con- cept taxonomy (Lin, 1998; Resnik, 1999). Anoth- er direction is to learn semantic similarity with the help of large corpus such as Web or Wikipedia da- ta (Sahami and Heilman, 2006; Yih and Meek, 2007; Bollegala et al., 2011; Gabrilovich and Markovitch, 2007). The basic assumption of this kind of methods is that the word with similar semantic meanings of- ten co-occur in the given corpus. Extensive experi- ments have validated the effectiveness of the corpus- based method in polarity classification task (Turney, 2002; Kaji and Kitsuregawa, 2007; Velikovich et al., 2010). For example, PMI is a well-known similari- ty measurement (Turney, 2002), which makes use of the whole Web as the corpus, and utilizes the search engine hits number to estimate the co-occurrence probability of the give word pairs. The PMI based method has achieved promising results. However, according to Kanayama’s investigation, only 60% co-occurrences in the same window in Web pages reflect the same sentiment orientation (Kanayama and Nasukawa, 2006). Therefore, we may ask the question whether the choosing of corpus can change the performance of sim and is there any better cor- pus than the Web page data for measuring the senti- ment similarity? Everyday, enormous numbers of tweets that con- tain people’s rich sentiments are …

Content truncated to 500 words