Towards Debugging Sentiment Lexicons

Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, pages 1024–1034, Beijing, China, July 26-31, 2015. c⃝2015 Association for Computational Linguistics Towards Debugging Sentiment Lexicons Andrew Schneider Computer and Information Sciences Temple University atschneider@temple.edu Eduard Dragut Computer and Information Sciences Temple University edragut@temple.edu Abstract Central to many sentiment analysis tasks are sentiment lexicons (SLs). SLs exhibit polarity inconsistencies. Previous work studied the problem of checking the con- sistency of an SL for the case when the en- tries have categorical labels (positive, neg- ative or neutral) and showed that it is NP- hard. In this paper, we address the more general problem, in which polarity tags take the form of a continuous distribution in the interval [0, 1]. We show that this problem is polynomial. We develop a gen- eral framework for addressing the consis- tency problem using linear programming (LP) theory. LP tools allow us to uncover inconsistencies efficiently, paving the way to building SL debugging tools. We show that previous work corresponds to 0-1 inte- ger programming, a particular case of LP. Our experimental studies show a strong correlation between polarity consistency in SLs and the accuracy of sentiment tag- ging in practice. 1 Introduction Many sentiment analysis algorithms rely on sen- timent lexicons (SLs), where word forms or word senses1 are tagged as conveying positive, negative or neutral sentiments. SLs are constructed by one of three methods (Liu, 2012; Feldman, 2013): (1) Manual tagging by human annotators is gener- ally reliable, but because it is labor-intensive, slow, and costly, this method has produced small-sized SLs comprising a few thousand words, e.g., Opin- ion Finder (OF) (Wilson et al., 2005), Appraisal Lexicon (AL) (Taboada and Grieve, 2004), Gen- eral Inquirer (GI) (Stone et al., 1966), and Micro- WNOp (Cerini et al., 2007). (2) Dictionary- 1We refer to a string of letters or sounds as a word form & to a pairing of a word form with a meaning as a word sense. based acquisition relies on a set of seed words to expand its coverage to similar words. There are over thirty dictionary-based techniques (An- dreevskaia and Bergler, 2006; Blum et al., 2004; Chen and Skiena, 2014; Choi and Wiebe, 2014; Esuli and Sebastiani, 2006; Feng et al., 2013; Has- san and Radev, 2010; Kamps et al., 2004; Moham- mad et al., 2009; Takamura et al., 2005; Turney, 2002; Williams and Anand, 2009), most of them based on WordNet (Fellbaum, 1998), such as Sen- tiWordNet (SWN)(Baccianella et al., 2010) and Q-WordNet (QWN) (Agerri and Garc´ıa-Serrano, 2010). (3) Corpus-based acquisition expands a set of seed words with the use of a large docu- ment corpus (Breck et al., 2007; Bross and Ehrig, 2013; Choi and Cardie, 2009; Ding et al., 2008; Du et al., 2010; Hatzivassiloglou and McKeown, 1997; Jijkoun et al., 2010; Kaji and Kitsuregawa, 2007; Klebanov et al., 2013; Lu et al., 2011; Peng and Park, 2011; Tang et al., 2014; Wu and Wen, …

Content truncated to 500 words