Multi-Level Structured Models for Document-Level Sentiment Classification

Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pages 1046–1056, MIT, Massachusetts, USA, 9-11 October 2010. c⃝2010 Association for Computational Linguistics Multi-level Structured Models for Document-level Sentiment Classification Ainur Yessenalina Dept. of Computer Science Cornell University Ithaca, NY, USA ainur@cs.cornell.edu Yisong Yue Dept. of Computer Science Cornell University Ithaca, NY, USA yyue@cs.cornell.edu Claire Cardie Dept. of Computer Science Cornell University Ithaca, NY, USA cardie@cs.cornell.edu Abstract In this paper, we investigate structured mod- els for document-level sentiment classifica- tion. When predicting the sentiment of a sub- jective document (e.g., as positive or nega- tive), it is well known that not all sentences are equally discriminative or informative. But identifying the useful sentences automatically is itself a difficult learning problem. This pa- per proposes a joint two-level approach for document-level sentiment classification that simultaneously extracts useful (i.e., subjec- tive) sentences and predicts document-level sentiment based on the extracted sentences. Unlike previous joint learning methods for the task, our approach (1) does not rely on gold standard sentence-level subjectivity an- notations (which may be expensive to obtain), and (2) optimizes directly for document-level performance. Empirical evaluations on movie reviews and U.S. Congressional floor debates show improved performance over previous ap- proaches. 1 Introduction Sentiment classification is a well-studied and active research area (Pang and Lee, 2008). One of the main challenges for document-level sentiment categoriza- tion is that not every part of the document is equally informative for inferring the sentiment of the whole document. Objective statements interleaved with the subjective statements can be confusing for learning methods, and subjective statements with conflicting sentiment further complicate the document catego- rization task. For example, authors of movie reviews often devote large sections to (largely objective) de- scriptions of the plot (Pang and Lee, 2004). In ad- dition, an overall positive review might still include some negative opinions about an actor or the plot. Early research on document-level sentiment clas- sification employed conventional machine learning techniques for text categorization (Pang et al., 2002). These methods, however, assume that documents are represented via a flat feature vector (e.g., a bag-of- words). As a result, their ability to identify and ex- ploit subjectivity (or other useful) information at the sentence-level is limited. And although researchers subsequently proposed methods for incorporating sentence-level subjectiv- ity information, existing techniques have some un- desirable properties. First, they typically require gold standard sentence-level annotations (McDon- ald et al. (2007), Mao and Lebanon (2006)). But the cost of acquiring such labels can be prohibitive. Second, some solutions for incorporating sentence- level information lack mechanisms for controlling how errors propagate from the subjective sentence identification subtask to the main document classifi- cation task (Pang and Lee, 2004). Finally, solutions that attempt to handle the error propagation problem have done so by explicitly optimizing for the best combination of document- and sentence-level clas- sification accuracy (McDonald et al., 2007). Opti- mizing for this compromise, when the real goal is to maximize only the document-level accuracy, can potentially hurt document-level performance. In …

Content truncated to 500 words