On the Automatic Learning of Sentiment Lexicons

Human Language Technologies: The 2015 Annual Conference of the North American Chapter of the ACL, pages 1397–1402, Denver, Colorado, May 31 – June 5, 2015. c⃝2015 Association for Computational Linguistics On the Automatic Learning of Sentiment Lexicons Aliaksei Severyn DISI, University of Trento 38123 Povo (TN), Italy severyn@disi.unitn.it Alessandro Moschitti Qatar Computing Research Institue 5825 Doha, Qatar amoschitti@qf.org.qa Abstract This paper describes a simple and princi- pled approach to automatically construct sen- timent lexicons using distant supervision. We induce the sentiment association scores for the lexicon items from a model trained on a weakly supervised corpora. Our empiri- cal findings show that features extracted from such a machine-learned lexicon outperform models using manual or other automatically constructed sentiment lexicons. Finally, our system achieves the state-of-the-art in Twitter Sentiment Analysis tasks from Semeval-2013 and ranks 2nd best in Semeval-2014 according to the average rank. 1 Introduction One of the early and rather successful models for sentiment analysis (Pang and Lee, 2004; Pang and Lee, 2008) relied on manually constructed lexicons that map words to their sentiment, e.g., positive, negative or neutral. The document-level polarity is then assigned by performing some form of averag- ing, e.g., majority voting, of individual word polari- ties found in the document. These systems show an acceptable level of accuracy, they are easy to build and are highly computationally efficient as the only operation required to assign a polarity label are the word lookups and averaging. However, the informa- tion about word polarities in a document are best ex- ploited when using machine learning models to train a sentiment classifier. In fact, most successful sentiment classification systems rely on supervised learning. Interestingly, a simple bag of words model using just unigrams and bigrams with an SVM has shown excellent re- sults (Wang and Manning, 2012) performing on par or beating more complicated models, e.g., using neural networks (Socher et al., 2011). Regarding Twitter sentiment analysis, the top performing system (Mohammad et al., 2013) from Semeval-2013 Twittter Sentiment Analysis task (Nakov et al., 2013) follows this recipe by train- ing an SVM on various surface form, sentiment and semantic features. Perhaps, the most valuable find- ing is that sentiment lexicons appear to be the most useful source of features accounting for over 8 point gains in the F-measure on top of the standard feature sets. Sentiment lexicons are mappings from words to scores capturing the degree of the sentiment ex- pressed by a given word. While several manually constructed lexicons are made available, e.g., the MPQA (Wilson et al., 2005), the Bing and Liu (Hu and Liu, 2004) and NRC Emoticon (Mohammad and Turney, 2013) lexicons, providing high quality word-sentiment associations compiled by humans, still their main drawback is low recall. For example, the largest NRC Emoticon lexicon contains only 14k items, whereas tweets with ex- tremely sparse surface forms are known to form very large vocabularies. Hence, using larger lexicons with better recall has the potential of learning more accurate models. Extracting such lexicons automat- ically is …

Content truncated to 500 words