Automatically Creating General-Purpose Opinion Summaries from Text

Proceedings of Recent Advances in Natural Language Processing, pages 202–209, Hissar, Bulgaria, 12-14 September 2011. Automatically Creating General-Purpose Opinion Summaries from Text Veselin Stoyanov Johns Hopkins University ves@cs.jhu.edu Claire Cardie Cornell University cardie@cs.cornell.edu Abstract We present and evaluate the first method known to us that can create rich non- extract-based opinion summaries from general text (e.g. newspaper articles). We first describe two possible representations for opinion summaries and then present our system OASIS, which identifies, and optionally aggregates, fine-grained opin- ions from the same source on the same topic. We propose new evaluation mea- sures for both types of opinion summary and employ the metrics in an evaluation of OASIS on a standard opinion corpus. Our results are encouraging — OASIS sub- stantially outperforms a competitive base- line when creating document-level aggre- gate summaries that compute the average polarity value across the multiple opin- ions identified for each source about each topic. We further show that as state-of- the-art performance on fine-grained opin- ion extraction improves, we can expect to see opinion summaries of very high qual- ity — with F-scores of 54-78% using our OSEM evaluation measure. 1 Introduction To date, most of the research in opinion analy- sis (see Related Work section) has focused on the problem of extracting opinions — both at the doc- ument level (coarse-grained opinion information) and at the level of sentences, clauses, or individual expressions (fine-grained opinion information). In contrast, our work concerns the consolidation of fine-grained information about opinions to cre- ate non-extract-based opinion summaries, a rich, concise and useful representation of the opinions expressed in a document. In particular, the opin- ion summaries produced by our system combine opinions from the same source and/or about the same topic and aggregate multiple opinions from the same source on the same topic in a meaningful way. A simple opinion summary is shown in Fig- ure 1. In the sample text, there are seven opinions expressed — two negative and one positive opin- ion from the American public on the war in Iraq, two negative opinions of Bush on withdrawal from Iraq, and so on. These are aggregated in the graph- based summary. We expect that this type of opin- ion summary, based on fine-grained opinion infor- mation, will be important for information analysis applications in any domain where the analysis of opinions and other subjective language is critical. Our notion of summary is fundamentally different from the extract-based textual summaries used of- ten in Natural Language Processing. We use the term non-extract-based summary to make that dis- tinction explicit, but also use opinion summary to refer to the summaries that we propose. In this paper, we present and evaluate OA- SIS (for Opinion Aggregation and SummarIzation System), the first system known to us that can pro- duce rich non-extract-based opinion summaries from general text.1 The system relies on automat- ically extracted fine-grained opinion information and constructs fully automatic opinion summaries in a form that can be easily presented to humans or queried by …

Content truncated to 500 words