Building a Sentiment Corpus of Tweets in Brazilian Portuguese

Building a Sentiment Corpus of Tweets in Brazilian Portuguese Henrico Bertini Brum, Maria das Grac¸as Volpe Nunes Interinstitutional Center for Computational Linguistics (NILC) Institute of Mathematical and Computer Sciences, University of S˜ao Paulo henrico.brum@usp.br, gracan@icmc.usp.br Abstract The large amount of data available in social media, forums and websites motivates researches in several areas of Natural Language Processing, such as sentiment analysis. The popularity of the area due to its subjective and semantic characteristics motivates research on novel methods and approaches for classification. Hence, there is a high demand for datasets on different domains and different languages. This paper introduces TweetSentBR, a sentiment corpus for Brazilian Portuguese manually annotated with 15.000 sentences on TV show domain. The sentences were labeled in three classes (positive, neutral and negative) by seven annotators, following literature guidelines for ensuring reliability on the annotation. We also ran baseline experiments on polar- ity classification using six machine learning classifiers, reaching 80.38% on F-Measure in binary classification and 64.87% when including the neutral class. We also performed experiments in similar datasets for polarity classification task in comparison to this corpus. Keywords: Sentiment Analysis, Corpus Annotation, Social Media 1. Introduction Sentiment Analysis (SA) became a popular area of Natural Language Processing in the last decade. The classification of semantic orientation of documents is a challenge for arti- ficial intelligence methods since it is based not only on the regular meaning of words, but also on their semantic role in the context and on the author’s intention. Furthermore, the amount of data available in blogs, social media posts and forums has created a great opportunity for researchers to build datasets for evaluating methods and studying new linguistic phenomena. Websites on e-commerce, movie reviews and hotel reserva- tions usually allow the user to provide an objective evalua- tion besides the written commentaries. This objective eval- uation (binary recommendation, star score, 10-point scale) can be a good feature for automatic labeling large datasets on semantic orientation, thus improving the resources for researches over the past decades (Pang et al., 2002; Pang and Lee, 2005; Blitzer et al., 2007). The limitation of this technique is the data available in this conditions. Social media, for example, is a large source of user opinions and evaluation (Pak and Paroubek, 2010), but the lack of an objective score attached to the posts demands a manual annotation in order to data become useful for SA, even though the data is enriched by linguistic phenomena such as expressions, slangs and irony. Manual annotation ends up being more expensive and time consuming, since it demands several guarantees of accu- racy, such as developing guidelines, training annotators and revising the annotation (Hovy and Lavid, 2010). In this paper we introduce TweetSentBR (TTsBR), a corpus manually annotated with data extracted from Twitter. The section 2. presents some related work on SA and corpus annotation. Section 3. presents the corpus and its proper- ties, such as the size, the annotation tags, the information on annotators and the process of data extraction. Section …

Content truncated to 500 words