Using Argument-based Features to Predict and Analyse Review Helpfulness

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1358–1363 Copenhagen, Denmark, September 7–11, 2017. c⃝2017 Association for Computational Linguistics Using Argument-based Features to Predict and Analyse Review Helpfulness Haijing Liu1,2, Yang Gao1, Pin Lv1, Mengxue Li1,2, Shiqiang Geng3, Minglan Li1,2 and Hao Wang1 1Institute of Software, Chinese Academy of Sciences 2University of Chinese Academy of Sciences 3Beijing Information Science and Technology University {haijing2015, gaoyang, lvpin, mengxue2015}@iscas.ac.cn gsqwxh@163.com,{minglan2015, wanghao}@iscas.ac.cn Abstract We study the helpful product reviews i- dentification problem in this paper. We observe that the evidence-conclusion dis- course relations, also known as argu- ments, often appear in product reviews, and we hypothesise that some argumen- t-based features, e.g. the percentage of argumentative sentences, the evidences- conclusions ratios, are good indicators of helpful reviews. To validate this hy- pothesis, we manually annotate arguments in 110 hotel reviews, and investigate the effectiveness of several combinations of argument-based features. Experiments suggest that, when being used togeth- er with the argument-based features, the state-of-the-art baseline features can enjoy a performance boost (in terms of F1) of 11.01% in average. 1 Introduction Product reviews have significant influences on po- tential customers’ opinions and their purchase de- cisions (Chatterjee, 2001; Chen et al., 2004; Del- larocas et al., 2004). Instead of reading a long list of reviews, customers usually are only will- ing to view a handful of helpful reviews to make their purchase decisions. In other words, helpful reviews have even greater influences on the po- tential customers’ decision-making processes and thus on the sales; as a result, the automatic identi- fication of helpful reviews has received consider- able research attentions in recent years (Kim et al., 2006; Liu et al., 2008; Mudambi, 2010; Xiong and Litman, 2014; Martin and Pu, 2014; Yang et al., 2015, 2016). Existing works on helpful reviews identifica- tion mostly focus on designing efficient features. Widely used features include external features, (e.g. date (Liu et al., 2008), product rating (Kim et al., 2006) and product type (Mudambi, 2010)) and intrinsic features (e.g. semantic dictionar- ies (Yang et al., 2015) and emotional dictionaries (Martin and Pu, 2014)). Compared to external fea- tures, intrinsic features can provide some insight- s and explanations for the prediction results, and support better cross-domain generalisation. In this work, we investigate a new form of intrinsic fea- tures: the argument features. An argument is a basic unit people use to per- suade their audiences to accept a particular state of affairs (Eckle-Kohler et al., 2015). An argu- ment usually consists of a claim (also known as conclusion) and some premises (also known as evidences) offered in support of the claim. For example, consider the following review excerpt: “The staff were amazing, they went out of their way to help us”; the texts before the comma con- stitute a claim, and the texts after the comma give a premise supporting the claim. Argumen- tation mining (Moens, 2013; Lippi and Torroni, 2016) receives growing research interests in var- ious domains (Palau and Moens, 2009; …

Content truncated to 500 words