Aspect-based summarization of pros and cons in unstructured product reviews

Proceedings of the 27th International Conference on Computational Linguistics, pages 2219–2229 Santa Fe, New Mexico, USA, August 20-26, 2018. 2219 Aspect-based summarization of pros and cons in unstructured product reviews Florian Kunneman Tilburg University PO Box 90153 5000LE Tilburg, The Netherlands f.a.kunneman@uvt.nl Sander Wubben Tilburg University PO Box 90153 5000LE Tilburg, The Netherlands s.wubben@uvt.nl Emiel Krahmer Tilburg University PO Box 90153 5000LE Tilburg, The Netherlands e.j.krahmer@uvt.nl Antal van den Bosch KNAW Meertens Institute Oudezijds Achterburgwal 185 1024DK Amsterdam, The Netherlands antal.van.den.bosch@meertens.knaw.nl Abstract We developed three systems for generating pros and cons summaries of product reviews. Au- tomating this task eases the writing of product reviews, and offers readers quick access to the most important information. We compared SynPat, a system based on syntactic phrases selected on the basis of valence scores, against a neural-network-based system trained to map bag-of- words representations of reviews directly to pros and cons, and the same neural system trained on clusters of word-embedding encodings of similar pros and cons. We evaluated the systems in two ways: first on held-out reviews with gold-standard pros and cons, and second by asking human annotators to rate the systems’ output on relevance and completeness. In the second eval- uation, the gold-standard pros and cons were assessed along with the system output. We find that the human-generated summaries are not deemed as significantly more relevant or complete than the SynPat systems; the latter are scored higher than the human-generated summaries on a precision metric. The neural approaches yield a lower performance in the human assessment, and are outperformed by the baseline. 1 Introduction Reviews posted on Web-based consumer platforms such as Amazon1 and Trustpilot2 form a valuable source of information prior to buying a product or hiring a service. Especially when a multitude of reviews of a specific product are gathered, the different experiences enable the consumer to obtain a well-informed conception of the qualities and deficiencies of the respective article. However, achieving a coherent view of a product tends to be challenging when a high volume of reviews diverge in focus and sentiment. A system that can summarize the reviews of a product would therefore be a valuable asset to these platforms. This work describes the implementation and evaluation of both a supervised and an unsupervised approach to aspect-based sentiment analysis, with a focus on Dutch product reviews of electric devices. We create summaries that zoom in on the pros and cons of the product that are mentioned in a re- view. Platforms that include product reviews commonly employ a pros and cons template to enable the authors of reviews themselves to summarize their review text. Although such a procedure could lead to This work is licensed under a Creative Commons Attribution 4.0 International License. License details: http:// creativecommons.org/licenses/by/4.0/ 1https://www.amazon.com/ 2https://www.trustpilot.com/ 2220 appropriate summaries, only a minority of users make the effort to fill in the pros and cons.3 On the one hand, this behavior underlines the need for a tool to generate a summary when the writer chooses …

Content truncated to 500 words