Improving Document-Level Sentiment Analysis with User and Product Context

Proceedings of the 28th International Conference on Computational Linguistics, pages 6724–6729 Barcelona, Spain (Online), December 8-13, 2020 6724 Improving Document-Level Sentiment Analysis with User and Product Context Chenyang Lyu School of Computing Dublin City University Dublin, Ireland chenyang.lyu2@mail.dcu.ie Jennifer Foster School of Computing Dublin City University Dublin, Ireland jennifer.foster@dcu.ie Yvette Graham School of Computer Science & Statistics Trinity College Dublin Dublin, Ireland ygraham@tcd.ie Abstract Past work that improves document-level sentiment analysis by encoding user and product in- formation has been limited to considering only the text of the current review. We investigate incorporating additional review text available at the time of sentiment prediction that may prove meaningful for guiding prediction. Firstly, we incorporate all available historical review text belonging to the author of the review in question. Secondly, we investigate the inclusion of his- torical reviews associated with the current product (written by other users). We achieve this by explicitly storing representations of reviews written by the same user and about the same product and force the model to memorize all reviews for one particular user and product. Additionally, we drop the hierarchical architecture used in previous work to enable words in the text to directly attend to each other. Experiment results on IMDB, Yelp 2013 and Yelp 2014 datasets show improvement to state-of-the-art of more than 2 percentage points in the best case. 1 Introduction Document-level sentiment analysis aims to predict sentiment polarity of text that often takes the form of product or service reviews. Tang et al. (2015) demonstrated that modelling the individual who has written the review, as well as the product being reviewed, is worthwhile for polarity prediction, and this has led to exploratory work on how best to combine review text with user/product information in a neural architecture (Chen et al., 2016; Ma et al., 2017; Dou, 2017; Long et al., 2018; Amplayo, 2019; Amplayo et al., 2018). A feature common amongst past studies is that user and product IDs are modelled as embedding vectors whose parameters are learned during training. We take this idea a step further and represent users and products using the text of all the reviews belonging to a single user or product – see Fig. 1 (left). There are two reasons to incorporate review text into user/product modelling. Firstly, the reviews from a given user will reflect their word choices when conveying sentiment. For example, a typical user might use words such as fantastic or excellent with correspondingly high ratings but another user could use the same words sarcastically with a low rating. Similarly, a group of users writing a review of the same product may use the same or similar opinionated words to refer to that product. Secondly, learning meaningful user and product embeddings that are only updated by back propagation is difficult when a user or product only has a small number of reviews, whereas one may still be able to glean something useful from the text of even a small number of reviews. A naive approach might …

Content truncated to 500 words