Detection of Product Comparisons - How Far Does an Out-of-the-Box Semantic Role Labeling System Take You?

Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1892–1897, Seattle, Washington, USA, 18-21 October 2013. c⃝2013 Association for Computational Linguistics Detection of Product Comparisons – How Far Does an Out-of-the-box Semantic Role Labeling System Take You? Wiltrud Kessler and Jonas Kuhn Institute for Natural Language Processing University of Stuttgart wiltrud.kessler@ims.uni-stuttgart.de Abstract This short paper presents a pilot study in- vestigating the training of a standard Seman- tic Role Labeling (SRL) system on product reviews for the new task of detecting com- parisons. An (opinionated) comparison con- sists of a comparative “predicate” and up to three “arguments”: the entity evaluated posi- tively, the entity evaluated negatively, and the aspect under which the comparison is made. In user-generated product reviews, the “predi- cate” and “arguments” are expressed in highly heterogeneous ways; but since the elements are textually annotated in existing datasets, SRL is technically applicable. We address the interesting question how well training an out- of-the-box SRL model works for English data. We observe that even without any feature en- gineering or other major adaptions to our task, the system outperforms a reasonable heuristic baseline in all steps (predicate identification, argument identification and argument classifi- cation) and in three different datasets. 1 Introduction Sentiment analysis deals with the task of determin- ing the polarity of an opinionated document or a sentence, in product reviews typically with regard to some target product. A common way to express sentiment about some product is by comparing it to a different product. In the corpus data we use, around 10% of sentences contain at least one comparison. Here are some examples of comparison sentences from our corpus: (1) a. “[This camera]E+ ... its [screen]A is much big- ger than the [400D].” b. “[D70]E+ beats [EOS 300D]E− in almost [ev- ery category]A, EXCEPT ONE.” c. “[Noise suppression]A1A2 was generally better1 than the [D80]E−1’s and much better2 than the [Rebel]E−2’s.” d. “A striking difference between the [EOS 350D]E− and the new [EOS 400D]E+ concerns the [image sensor]A.” Note that our definition of comparisons is broader than the linguistic category of comparative sen- tences, which only includes sentences that contain a comparative adjective or adverb. For our work, we consider comparisons expressed by any Part of Speech (POS). A comparison contains several parts that must be identified in order to get meaningful information. We call the word or phrase that is used to express the comparison (“better”, “beats”, ...) a comparative predicate. A comparison involves two entities, one or both of them may be implicit. In our data, most of the entities are products, e.g., the two cameras “D70” and “EOS 300D” in sentence 1b. In graded comparisons, entity+ (E+) is the entity that is being evaluated positively, entity- (E-) the entity evaluated negatively. In many sentences one attribute or part of a product is being compared, like “image sensor” in sentence 1d. We call this the aspect (A). The task we want to solve for a given compari- son sentence is to detect …

Content truncated to 500 words