Using Multiple Sources of Agreement Information for Sentiment Classification of Political Transcripts

Using multiple sources of agreement information for sentiment classification of political transcripts Clint Burfoot Department of Computer Science and Software Engineering University of Melbourne, VIC 3010, Australia c.burfoot@pgrad.unimelb.edu.au Abstract Sentiment classifiers attempt to determine whether a document expresses a generally positive or negative sentiment about its topic. Previous work has shown that overall per- formance can be improved by combining per-document classifications with information about agreement between documents. This paper explores approaches to sentiment clas- sification of U.S Congressional floor debate transcripts that use a model for incorporating multiple sources of information about agree- ments between speakers. An empirical eval- uation demonstrates accuracy improvements over previously published results. 1 Introduction Thomas et al. (2006) investigate whether one can de- termine from the transcripts of U.S. Congressional floor debates whether speeches support or oppose a piece of proposed legislation. The authors ex- ploit the fact that speeches occur as part of a discus- sion in order to incorporate information about their inter-relationships. Using this information provides substantial improvements over analysing speeches in isolation. The authors adopt a model, based on graph mincuts, that optimally balances classifica- tions of speeches performed in isolation with infor- mation about whether or not pairs of speeches are likely to agree. The work reported here mirrors Thomas et al. in using the same corpus and attempting the same binary sentiment classification task. It refines the the concept of agreement, comparing alternate tech- niques for detecting agreement in debates, and showing how these techniques can improve perfor- mance in the task of classifying speeches. Two types of agreement information are intro- duced. The first, party agreement, takes advantage of the fact that speakers tend on average to vote with the majority of their party. A party-classifier is trained to detect whether a speaker is most likely a Republican or a Democrat. Links between pairs of speeches assigned to the same party encourage the overall classifier to give these speeches the same label. The second type of agreement is based on the intuition that speakers who agree often use sim- ilar words to describe their position. This similar- ity agreement is derived by comparing the context windows that surround references to the bill being considered in the debate. An experimental evaluation shows that incremen- tal accuracy improvements can be gained by making use of this agreement information. A range of tech- nical limitations are discussed and ideas considered for future work that may make even better use of agreement information to improve sentiment classi- fication. 2 Related Work For a detailed survey of the field of sentiment analy- sis see (Pang and Lee, 2008). The document-level sentiment-polarity classifica- tion task began as a “thumbs up or thumbs down” ex- periment. Pang et al. (2002) apply machine learning methods to predicting the overall sentiment of movie reviews. Turney uses PMI-IR to perform sentiment classification of reviews of banks, movies, cars and travel destinations (Turney, 2002; Turney, 2001). Much work has focussed on the problem of sep- arating language that …

Content truncated to 500 words