Identifying Helpful Sentences in Product Reviews

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 678–691 June 6–11, 2021. ©2021 Association for Computational Linguistics 678 Identifying Helpful Sentences in Product Reviews Iftah Gamzu1 Hila Gonen1 Gilad Kutiel1 Ran Levy1 Eugene Agichtein2,3 {iftah,gonenhi,gkutiel,ranlevy,eugeneag}@amazon.com 1Amazon, Tel-Aviv, Israel 2Amazon, Seattle, WA, USA 3Emory University, GA, USA Abstract In recent years online shopping has gained momentum and became an important venue for customers wishing to save time and sim- plify their shopping process. A key advan- tage of shopping online is the ability to read what other customers are saying about prod- ucts of interest. In this work, we aim to main- tain this advantage in situations where extreme brevity is needed, for example, when shop- ping by voice. We suggest a novel task of extracting a single representative helpful sen- tence from a set of reviews for a given prod- uct. The selected sentence should meet two conditions: first, it should be helpful for a pur- chase decision and second, the opinion it ex- presses should be supported by multiple re- viewers. This task is closely related to the task of Multi Document Summarization in the product reviews domain but differs in its ob- jective and its level of conciseness. We col- lect a dataset in English of sentence helpful- ness scores via crowd-sourcing and demon- strate its reliability despite the inherent subjec- tivity involved. Next, we describe a complete model that extracts representative helpful sen- tences with positive and negative sentiment to- wards the product and demonstrate that it out- performs several baselines. 1 Introduction Customer reviews are known to be a valuable source of information for potential buyers. This is evident from the high engagement of customers with reviews, for example by up-voting a review for its helpfulness.1 As online shopping platforms attract more traffic it is becoming increasingly dif- ficult to consume the wealth of information cus- tomers share. For this reason, helpful reviews (de- fined as such by costumers) are made more visible than those that are less helpful. 1At the time of writing this paper, a review for the Echo Dot 3rd generation received more than 10K votes. The topic of review helpfulness has attracted a lot of academic interest in which reviews were always considered as a whole (see Diaz and Ng (2018) for a survey). However, in some scenarios, such as the limited real-estate in mobile screens, or in voice interactions with a virtual assistant, pre- senting a full review is impractical and the need to automatically extract helpful excerpts arises. While in the mobile scenario, a persistent customer may still be able to read the entire review, the voice sce- nario is inherently more challenging as it demands patience and focus from the customer, while the as- sistant reads the text out loud. As a result, the need for extreme brevity and the ability to understand what matters most to customers becomes crucial. In addition to brevity and helpfulness, another …

Content truncated to 500 words