Empirical analysis of exploiting review helpfulness for extractive summarization of online reviews

Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, pages 1985–1995, Dublin, Ireland, August 23-29 2014. Empirical analysis of exploiting review helpfulness for extractive summarization of online reviews Wenting Xiong University of Pittsburgh Department of Computer Science wex12@cs.pitt.edu Diane Litman University of Pittsburgh Department of Computer Science & LRDC litman@cs.pitt.edu Abstract We propose a novel unsupervised extractive approach for summarizing online reviews by ex- ploiting review helpfulness ratings. In addition to using the helpfulness ratings for review-level filtering, we suggest using them as the supervision of a topic model for sentence-level content scoring. The proposed method is metadata-driven, requiring no human annotation, and generaliz- able to different kinds of online reviews. Our experiment based on a widely used multi-document summarization framework shows that our helpfulness-guided review summarizers significantly outperform a traditional content-based summarizer in both human evaluation and automated eval- uation. 1 Introduction Multi-document summarization has great potential in online reviews, as manually reading comments provided by other users is time consuming if not impossible. While extractive techniques are generally preferred over abstractive ones (as abstraction can introduce disfluency), existing extractive summarizers are either supervised or based on heuristics of certain desired characteristics of the summarization result (e.g., maximize n-gram coverage (Nenkova and Vanderwende, 2005), etc.). However, when it comes to online reviews, there are problems with both approaches: the first one requires manual annotation and is thus less generalizable; the second one might not capture the salient information in reviews from different domains (camera reviews vs. movie reviews), because the heuristics are designed for traditional genres (e.g., news articles) while the utility of reviews might vary with the review domain. We propose to exploit review metadata, that is review helpfulness ratings1, to facilitate review sum- marization. Because this is user-provided feedback on review helpfulness which naturally reflects users’ interest in online review exploration, our approach captures domain-dependent salient information adap- tively. Furthermore, as this metadata is widely available online (e.g., Amazon.com, IMDB.com)2, our approach is unsupervised in the sense that no manual annotation is needed for summarization purposes. Therefore, we hypothesize that summarizers guided by review helpfulness will outperform systems based on textual features/heuristics designed for traditional genres. To build such helpfulness-guided summa- rizers, we introduce review helpfulness during content selection in two ways: 1) using the review-level helpfulness ratings directly to filter out unhelpful reviews, 2) using sentence-level helpfulness features derived from review-level helpfulness ratings for sentence scoring. As we observe in our pilot study that supervised LDA (sLDA) (Blei and McAuliffe, 2010) trained with review helpfulness ratings has potential in differentiating review helpfulness at the sentence level, we develop features based on the inferred hidden topics from sLDA to capture the helpfulness of a review sentence for summarization pur- poses. We implement our helpfulness-guided review summarizers based on an widely used open-source multi-document extractive summarization framework (MEAD (Radev et al., 2004)). Both human and This work is licensed under a Creative Commons Attribution 4.0 International Licence. Page numbers and proceedings footer are added …

Content truncated to 500 words