Ranking Online Reviews Based on Their Helpfulness: An Unsupervised Approach

Proceedings of Recent Advances in Natural Language Processing, pages 959–967 Sep 1–3, 2021. https://doi.org/10.26615/978-954-452-072-4_109 959 Ranking Online Reviews Based on Their Helpfulness: An Unsupervised Approach Alimuddin Melleng Anna-Jurek Loughrey Deepak P Queen’s University Belfast, UK alimuddinmllg@gmail.com, a.jurek@qub.ac.uk, deepaksp@acm.org Abstract Online reviews are an essential aspect of on- line shopping for both customers and retail- ers. However, many reviews found on the In- ternet lack in quality, informativeness or help- fulness. In many cases, they lead the cus- tomers towards positive or negative opinions without providing any concrete details (e.g., very poor product, I would not recommend it). In this work, we propose a novel unsuper- vised method for quantifying helpfulness lever- aging the availability of a corpus of reviews. In particular, our method exploits three char- acteristics of the reviews, viz., relevance, emo- tional intensity and specificity, towards quanti- fying helpfulness. We perform three rankings (one for each feature above), which are then combined to obtain a final helpfulness ranking. For the purpose of empirically evaluating our method, we use review of four product cate- gories from Amazon review1. The experimen- tal evaluation demonstrates the effectiveness of our method in comparison to a recent and state-of-the-art baseline. 1 Introduction Reviews are an essential aspect of information that allows users to obtain insight into a product of interest before purchasing. Typically, users write their reviews in order to express their satisfaction or dissatisfaction about purchased items or services. Products and sellers with more positive reviews tend to gain more new customers than products or sellers without reviews or with many negative reviews. This is because customers feel more confi- dent buying products that have been recommended by other buyers. Popular products could have hun- dreds or thousands of reviews, which makes it impossible for the customers to read all of them. Moreover, it is not easy for a user to prioritize read- ing the most informative reviews since there are 1http://jmcauley.ucsd.edu/data/amazon/ often no such ranking options. Some websites rank reviews based on the posting date or rating star, for example, Trustpilot.com and Reviews.io. Amazon uses a crowdsourcing mechanism, a voting system, to gather feedback on review helpfulness, and then rank them based on the overall votes they received (Amazon.com). A user can vote for a review as being helpful or unhelpful. Amazon was estimated to receive a revenue of about $2.7 billion by pro- viding simple question “was this review helpful to you?” (Spool, 2009). Although such a voting system is helpful for cus- tomers, it has several limitations due to the inherent character of the voting process. There are number of reasons: 1) not all reviews get the helpfulness vote; 2) the helpfulness voting does not work for cold star review (i.e., a new user or a new review will have much less votes) (Singh et al., 2017); 3) reviews receiving helpfulness votes would tend to gather more vote due to the snowball effect (e.g., phenomena such as social proof (Cialdini, 1987)). In this work we hypothesise that helpfulness of …

Content truncated to 500 words