More than Bag-of-Words: Sentence-based Document Representation for Sentiment Analysis

Proceedings of Recent Advances in Natural Language Processing, pages 546–552, Hissar, Bulgaria, 7-13 September 2013. More than Bag-of-Words: Sentence-based Document Representation for Sentiment Analysis Georgios Paltoglou Faculty of Science and Technology University of Wolverhampton Wulfruna Street, WV1 1LY, UK g.paltoglou@wlv.ac.uk Mike Thelwall Faculty of Science and Technology University of Wolverhampton Wulfruna Street, WV1 1LY, UK m.thelwall@wlv.ac.uk Abstract Most sentiment analysis approaches rely on machine-learning techniques, using a bag-of-words (BoW) document represen- tation as their basis. In this paper, we examine whether a more fine-grained rep- resentation of documents as sequences of emotionally-annotated sentences can in- crease document classification accuracy. Experiments conducted on a sentence and document level annotated corpus show that the proposed solution, combined with BoW features, offers an increase in classi- fication accuracy. 1 Introduction Sentiment analysis is concerned with automat- ically extracting sentiment-related information from text. A typical problem is to determine whether a text is positive, negative or neutral over- all. Most of the proposed solutions are based on supervised machine-learning approaches, with some notable exceptions (Turney, 2002; Lin and He, 2009), although unsupervised, lexicon-based solutions have also been used, especially in non review-based corpora (Thelwall et al., 2010). This paper deals with the problem of detecting the overall polarity of a document. A common theme with a significant number of proposed solu- tions is the bag-of-words (BoW) document repre- sentation, according to which a document is repre- sented as a binary or frequency-based feature vec- tor of the tokens it contains, regardless of their po- sition in the text. Nonetheless, significant seman- tic information is lost when all positional informa- tion is discarded. Consider, the following extract of a movie review (taken from Pang (2008)): This film should be brilliant. It sounds like a great plot, . . . a good performance. However, it cant hold up. Most of bag-of-words machine learning or lexicon-based solutions would be expected to clas- sify the extract as positive because of the signif- icant number of positive words that it contains. However, a human reader studying the review, rec- ognizes the change of polarity that occurs in the last sentence, a change that is hinted at by the first sentence (“should be brilliant”) but is only fully realized at the end. In fact, this phenomenon of “thwarted expectations” is particularly common in reviews and has been observed by both Pang et. al (2002) and Turney (2002) who noted that “the whole is not necessarily the sum of the parts”. In this work we propose a solution to the afore- mentioned problem by building a meta-classifier which models each document as a sequence of emotionally annotated sentences. The advantage of this modeling is that it implicitly captures word position in the whole document in a semantically and structurally meaningful way, while at the same time drastically reducing the feature space for the final classification. Additionally, the proposed so- lution is conceptually simple, intuitive and can be used in addition to standard BoW features. 2 Prior Work The commercial potential of …

Content truncated to 500 words