Detecting Deceptive Opinions with Profile Compatibility

International Joint Conference on Natural Language Processing, pages 338–346, Nagoya, Japan, 14-18 October 2013. Detecting deceptive opinions with profile compatibility Vanessa Wei Feng University of Toronto Toronto, ON, M5S 3G4, Canada weifeng@cs.toronto.edu Graeme Hirst University of Toronto Toronto, ON, M5S 3G4, Canada gh@cs.toronto.edu Abstract We propose using profile compatibility to differentiate genuine and fake product re- views. For each product, a collective profile is derived from a separate col- lection of reviews. Such a profile con- tains a number of aspects of the prod- uct, together with their descriptions. For a given unseen review about the same product, we build a test profile using the same approach. We then perform a bidi- rectional alignment between the test and the collective profile, to compute a list of aspect-wise compatible features. We adopt Ott et al. (2011)’s op spam v1.3 dataset for identifying truthful vs. decep- tive reviews. We extend the recently pro- posed N-GRAM+SYN model of Feng et al. (2012a) by incorporating profile compat- ibility features, showing such an addition significantly improves upon their state-of- art classification performance. 1 Introduction With the rapid development of e-commerce and the increasing popularity of various product re- view websites, people are more and more used to making purchase decisions based on the reported experience of other customers. A product rated positively by its previous users is able to attract po- tential new customers, while a poorly rated prod- uct is certainly not a good option for most new customers. Given this influential power of prod- uct reviews, there comes a huge potential for de- ceptive opinion spam to distort the true evaluation of a product. The promoters of a product may post false complimentary reviews, and competi- tors may post false derogatory reviews. Although the task of detecting deceptive opin- ion spam can be formulated as a traditional bi- nary classification problem with two classes, de- ceptive and truthful, where supervised learning can be applied, it is essentially very challenging. Its major difficulty arises from the lack of reli- ably labeled data, because it is extremely difficult for humans to identify through reading (see dis- cussion in Section 4). Therefore, much previous work on detecting deceptive opinions usually re- lies on some meta-information, such as the IP ad- dress of the reviewer or the average rating of the product, rather than the actual content of the re- view Liu (2012). However, thanks to the release of the op spam v1.3 dataset by Ott et al. (2011), a gold-standard dataset composed of 400 truthful and 400 deceptive hotel reviews (see Section 3), we are now at an appropriate stage for conducting supervised learning and reliable evaluation of the task. In their work, Ott et al. proposed to use n-grams as features, and trained an SVM classifier to clas- sify whether a given review is deceptive or truth- ful, achieving nearly 90% accuracy. More re- cently, Feng et al. (2012a) extended Ott et al.’s n-gram feature set by incorporating deep syntax features, i.e., syntactic production rules …

Content truncated to 500 words