Incorporating Content Structure into Text Analysis Applications

Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pages 377–387, MIT, Massachusetts, USA, 9-11 October 2010. c⃝2010 Association for Computational Linguistics Incorporating Content Structure into Text Analysis Applications Christina Sauper, Aria Haghighi, Regina Barzilay Computer Science and Artificial Intelligence Laboratory Massachusetts Institute of Technology {csauper, aria42, regina}@csail.mit.edu Abstract In this paper, we investigate how modeling content structure can benefit text analysis ap- plications such as extractive summarization and sentiment analysis. This follows the lin- guistic intuition that rich contextual informa- tion should be useful in these tasks. We present a framework which combines a su- pervised text analysis application with the in- duction of latent content structure. Both of these elements are learned jointly using the EM algorithm. The induced content struc- ture is learned from a large unannotated cor- pus and biased by the underlying text analysis task. We demonstrate that exploiting content structure yields significant improvements over approaches that rely only on local context.1 1 Introduction In this paper, we demonstrate that leveraging doc- ument structure significantly benefits text analysis applications. As a motivating example, consider the excerpt from a DVD review shown in Table 1. This review discusses multiple aspects of a product, such as audio and video properties. While the word “pleased” is a strong indicator of positive sentiment, the sentence in which it appears does not specify the aspect to which it relates. Resolving this ambiguity requires information about global document struc- ture. A central challenge in utilizing such informa- tion lies in finding a relevant representation of con- tent structure for a specific text analysis task. For 1Code and processed data presented here are available at http://groups.csail.mit.edu/rbg/code/content structure.html Audio Audio choices are English, Spanish and French Dolby Digital 5.1 ... Bass is still robust and powerful, giving weight to just about any scene – most notably the film’s exciting final fight. Fans should be pleased with the presentation. Extras This single-disc DVD comes packed in a black amaray case with a glossy slipcover. Cover art has clearly been designed to appeal the Twilight crowd ... Finally, we’ve got a deleted scenes reel. Most of the excised scenes are actually pretty interesting. Table 1: An excerpt from a DVD review. instance, when performing single-aspect sentiment analysis, the most relevant aspect of content struc- ture is whether a given sentence is objective or sub- jective (Pang and Lee, 2004). In a multi-aspect setting, however, information about the sentence topic is required to determine the aspect to which a sentiment-bearing word relates (Snyder and Barzi- lay, 2007). As we can see from even these closely re- lated applications, the content structure representa- tion should be intimately tied to a specific text anal- ysis task. In this work, we present an approach in which a content model is learned jointly with a text analy- sis task. We assume complete annotations for the task itself, but we learn the content model from raw, unannotated text. Our approach is implemented in a discriminative framework using latent …

Content truncated to 500 words