Towards a General Rule for Identifying Deceptive Opinion Spam

Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, pages 1566–1576, Baltimore, Maryland, USA, June 23-25 2014. c⃝2014 Association for Computational Linguistics Towards a General Rule for Identifying Deceptive Opinion Spam Jiwei Li1, Myle Ott2, Claire Cardie2, Eduard Hovy1 1Language Technology Institute, Carnegie Mellon University, Pittsburgh, P.A. 15213, USA 2Department of Computer Science, Cornell University, Ithaca, N.Y., 14853, USA bdlijiwei@gmail.com, myleott@cs.cornell.edu cardie@cs.cornell.edu, ehovy@andrew.cmu.edu Abstract Consumers’ purchase decisions are in- creasingly influenced by user-generated online reviews. Accordingly, there has been growing concern about the poten- tial for posting deceptive opinion spam— fictitious reviews that have been deliber- ately written to sound authentic, to de- ceive the reader. In this paper, we ex- plore generalized approaches for identify- ing online deceptive opinion spam based on a new gold standard dataset, which is comprised of data from three different do- mains (i.e. Hotel, Restaurant, Doctor), each of which contains three types of re- views, i.e. customer generated truthful re- views, Turker generated deceptive reviews and employee (domain-expert) generated deceptive reviews. Our approach tries to capture the general difference of language usage between deceptive and truthful re- views, which we hope will help customers when making purchase decisions and re- view portal operators, such as TripAdvisor or Yelp, investigate possible fraudulent ac- tivity on their sites.1 1 Introduction Consumers increasingly rely on user-generated online reviews when making purchase deci- sion (Cone, 2011; Ipsos, 2012). Unfortunately, the ease of posting content to the Web, poten- tially anonymously, creates opportunities and in- centives for unscrupulous businesses to post de- ceptive opinion spam—fictitious reviews that are deliberately written to sound authentic, in order to deceive the reader.2 Accordingly, there appears 1Dataset available by request from the first author. 2Manipulating online reviews may also have legal conse- quences. For example, the Federal Trade Commission (FTC) to be widespread and growing concern among both businesses and the public about this poten- tial abuse (Meyer, 2009; Miller, 2009; Streitfeld, 2012; Topping, 2010; Ott, 2013). Existing approaches for spam detection are usu- ally focused on developing supervised learning- based algorithms to help users identify decep- tive opinion spam, which are highly dependent upon high-quality gold-standard labeled data (Jin- dal and Liu, 2008; Jindal et al., 2010; Lim et al., 2010; Wang et al., 2011; Wu et al., 2010). Stud- ies in the literature rely on a couple of approaches for obtaining labeled data, which usually fall into two categories. The first relies on the judge- ments of human annotators (Jindal et al., 2010; Mukherjee et al., 2012). However, recent stud- ies show that deceptive opinion spam is not eas- ily identified by human readers (Ott et al., 2011). An alternative approach, as introduced by Ott et al. (2011), crowdsourced deceptive reviews using Amazon Mechanical Turk.3 A couple of follow-up works have been introduced based on Ott et al.’s dataset, including estimating prevalence of decep- tion in online reviews (Ott et al., 2012), identifica- tion of negative deceptive opinion spam (Ott et al., 2013), and identifying manipulated offerings (Li …

Content truncated to 500 words