Evaluating the Robustness of EmotiBlog for Sentiment Analysis and Opinion Mining

Proceedings of Recent Advances in Natural Language Processing, pages 521–526, Hissar, Bulgaria, 12-14 September 2011. Evaluating the Robustness of EmotiBlog for Sentiment Analysis and Opinion Mining Ester Boldrini, Javi Fernández, José Manuel Gómez and Patricio Martínez-Barco GPLSI – University of Alicante {eboldrini; javifm; jmgomez; patricio}@dlsi.ua.es Abstract Preliminary research demonstrated the Emo- tiBlog annotated corpus relevance as a Ma- chine Learning resource to detect subjective data. In this paper we compare EmotiBlog with the JRC Quotes corpus in order to check the robustness of its annotation. We concen- trate on its coarse-grained labels and carry out a deep Machine Learning experimentation also with the inclusion of lexical resources. The re- sults obtained show a similarity with the ones obtained with the JRC Quotes corpus demon- strating the EmotiBlog validity as a resource for the SA task. 1 Introduction and Motivation Due to the birth of the Web 2.0 and the wide employment of the new textual genres we have an exponential increase of the subjective infor- mation. We also have a recent explosion of inter- est in Sentiment Analysis (SA), a subtask of Natural Language Processing (NLP), in charge of identifying the opinions related to a specific tar- get (Liu, 2006). Subjective data has a great po- tential; it can be exploited by business organizations or individuals, for ads placements, but also for the Opinion Retrieval/Search, etc (Liu, 2007). Our research is motivated by the lack of resources, methods and tools to effec- tively process subjective information. Our main purpose is to demonstrate that the EmotiBlog corpus can be a robust resource to overcome the challenges SA brings. For these first experiments we take into account its coarse-grained annota- tion; however in the future we will concentrate on the finer-grained annotation. We train our Machine Learning (ML) system with EmotiBlog Kyoto1 and EmotiBlog Phones2 corpora, but also 1 The EmotiBlog corpus is composed by blog posts on the Kyoto Protocol, Elections in Zimbabwe and USA election, but for this research we only use the EmotiBlog Kyoto (about the Kyoto Protocol) 2 it is an EmotiBlog extension with reviews of mobiles with the JRC Quotes3 collection. These experi- ments are possible since the corpora share some common annotated elements (Section 3), thus allowing a larger dataset and comparable results. Then, we train our system with some of the fea- tures of EmotiBlog and we also integrate 2 lexi- cal resources to reach a wider coverage. We also employ NLP techniques (stemmer, lemmatiser, bag of words, etc.) to improve the results ob- tained with the supervised ML models. In previ- ous works it has been demonstrated that EmotiBlog is a beneficial resource for Opinion- ated Question Answering (OQA) as stated Bala- hur et al. (2009c and 2010) or Automatic Opinionated Summarization (Balahur et al. 2009a). Thus, our first objective is to demon- strate that EmotiBlog is a useful resource to train ML systems for SA. The combination of training from EmotiBlog and JRC Quotes is beneficial since it provides more data for the …

Content truncated to 500 words