Fine-Grained Sentiment Analysis for Movie Reviews in Bulgarian

Proceedings of Recent Advances in Natural Language Processing, pages 266–274, Hissar, Bulgaria, Sep 7–9 2015. Fine-Grained Sentiment Analysis for Movie Reviews in Bulgarian Borislav Kapukaranov Faculty of Mathematics and Informatics Sofia University Bulgaria b.kapukaranov@gmail.com Preslav Nakov Qatar Computing Research Institute HBKU Qatar pnakov@qf.org.qa Abstract We present a system for fine-grained senti- ment analysis in Bulgarian movie reviews. As this is pioneering work for this com- bination of language and sentiment gran- ularity, we create suitable, freely available resources: a dataset of movie reviews with fine-grained scores, and a sentiment po- larity lexicon. We further compare ex- perimentally the performance of classifi- cation, regression and ordinal regression in a 3-way, 5-way and 11-way classifi- cation setups, using as features not only the text from the reviews, but also con- textual information in the form of meta- data, e.g., movie length, director, actors, genre, country, and various scores: IMDB, Cinexio, and user-average. The results show that adding contextual information yields strong performance gains. 1 Introduction With the recent explosion in the popularity of Web forums and social media, sentiment anal- ysis has emerged as a hot research topic. As sentiment-annotated data became readily avail- able, researchers tapped into it and started devel- oping various models for sentiment polarity pre- diction. Nowadays, there are many applications for sentiment analysis, e.g., businesses getting au- tomatically classified feedback from customers, automated review scoring in retail Web sites, ex- ploration of positive and negative trends, etc. Movie reviews are a popular and widely avail- able source of sentiment-annotated data. Unlike reviews produced by critics, those contributed by users are typically short and serve primarily to pro- vide brief justification of a user’s rating. An im- portant characteristic of movie reviews compared to other sentiment sources is that they are com- monly scored on a 5-star scale. This is very different from sentiment analy- sis on Twitter, where three-way sentiment clas- sification schemes (positive, negative, neutral) have been preferred, e.g., at SemEval 2013-2015 (Nakov et al., 2013; Rosenthal et al., 2014; Rosen- thal et al., 2015; Nakov et al., 2015). In contrast, the star system makes the task more fine-grained, thus allowing to capture user opinion better. Here, we present experiments in predicting fine-grained stars, including halves, for Bulgarian movie reviews. This is a challenging task, that can be seen as (a) multi-way classification, i.e., choosing one out of eleven classes, (b) regression, i.e., predicting a real number, or (c) something in between, namely ordinal regression, i.e., pre- dicting eleven values, but taking ordering into ac- count, e.g., predicting 4 when the actual value is 3.5 would be better than predicting 1. While sentiment classification in movie reviews has been extensively studied for English (movie reviews datasets were among the earliest to use for this task), it has not been tried for Bulgarian so far. Moreover, most research has focused on positive/negative/neutral classification and finer- grained schemes have been less popular (as they are harder). Even when used, the focus has typ- ically been on having just …

Content truncated to 500 words