Improving the Impact of Subjectivity Word Sense Disambiguation on Contextual Opinion Analysis

Proceedings of the Fifteenth Conference on Computational Natural Language Learning, pages 87–96, Portland, Oregon, USA, 23–24 June 2011. c⃝2011 Association for Computational Linguistics Improving the Impact of Subjectivity Word Sense Disambiguation on Contextual Opinion Analysis Cem Akkaya, Janyce Wiebe, Alexander Conrad University of Pittsburgh Pittsburgh PA, 15260, USA {cem,wiebe,conrada}@cs.pitt.edu Rada Mihalcea University of North Texas Denton TX, 76207, USA rada@cs.unt.edu Abstract Subjectivity word sense disambiguation (SWSD) is automatically determining which word instances in a corpus are being used with subjective senses, and which are being used with objective senses. SWSD has been shown to improve the performance of contextual opinion analysis, but only on a small scale and using manually developed integration rules. In this paper, we scale up the integration of SWSD into contextual opinion analysis and still obtain improvements in performance, by successfully gathering data annotated by non-expert annotators. Further, by improving the method for integrating SWSD into con- textual opinion analysis, even greater benefits from SWSD are achieved than in previous work. We thus more firmly demonstrate the potential of SWSD to improve contextual opinion analysis. 1 Introduction Often, methods for opinion, sentiment, and sub- jectivity analysis rely on lexicons of subjective (opinion-carrying) words (e.g., (Turney, 2002; Whitelaw et al., 2005; Riloff and Wiebe, 2003; Yu and Hatzivassiloglou, 2003; Kim and Hovy, 2004; Bloom et al., 2007; Andreevskaia and Bergler, 2008; Agarwal et al., 2009)). Examples of such words are the following (in bold): (1) He is a disease to every team he has gone to. Converting to SMF is a headache. The concert left me cold. That guy is such a pain. However, even manually developed subjectiv- ity lexicons have significant degrees of subjectivity sense ambiguity (Su and Markert, 2008; Gyamfi et al., 2009). That is, many clues in these lexicons have both subjective and objective senses. This ambiguity leads to errors in opinion and sentiment analysis, be- cause objective instances represent false hits of sub- jectivity clues. For example, the following sentence contains the keywords from (1) used with objective senses: (2) Early symptoms of the disease include severe headaches, red eyes, fevers and cold chills, body pain, and vomiting. Recently, in (Akkaya et al., 2009), we introduced the task of subjectivity word sense disambiguation (SWSD), which is to automatically determine which word instances in a corpus are being used with sub- jective senses, and which are being used with objec- tive senses. We developed a supervised system for SWSD, and exploited the SWSD output to improve the performance of multiple contextual opinion anal- ysis tasks. Although the reported results are promising, there are three obvious shortcomings. First, we were able to apply SWSD to contextual opinion analysis only on a very small scale, due to a shortage of anno- tated data. While the experiments show that SWSD improves contextual opinion analysis, this was only on the small amount of opinion-annotated data that was in the coverage of our system. Two questions arise: is it feasible to obtain greater amounts of the needed data, and …

Content truncated to 500 words