Resource Creation Towards Automated Sentiment Analysis in Telugu (a low resource language) and Integrating Multiple Domain Sources to Enhance Sentiment Prediction

Resource Creation Towards Automated Sentiment Analysis in Telugu (a low resource language) and Integrating Multiple Domain Sources to Enhance Sentiment Prediction Gangula Rama Rohit Reddy, Radhika Mamidi LTRC IIIT Hyderabad, LTRC IIIT Hyderabad Hyderabad Telangana India, Hyderabad Telangana India ramarohitreddy.g@research.iiit.ac.in, radhika.mamidi@iiit.ac.in Abstract Understanding the polarity or sentiment of a text is an important task in many application scenarios. Sentiment Analysis of a text can be used to answer various questions such as election prediction, favouredness towards any product etc. But the sentiment analysis task becomes challenging when it comes to low resource languages because the basis of learning sentiment classifiers are annotated datasets and annotated datasets for non-English texts hardly exists. So for the development of sentiment classifiers in Telugu, we have created corpora "Sentiraama" for different domains like movie reviews, song lyrics, product reviews and book reviews in Telugu language with the text written in Telugu script. In this paper, we describe the process of creating the corpora and assigning polarities to them. After the creation of corpora, we trained the classifiers that yields good classification results. Typically a sentiment classifier is trained using data from the same domain it is intended to be tested on. But there may not be sufficient data available in the same domain and additionally using data from multiple sources and domains may help in creating a more generalized sentiment classifier which can be applied to multiple domains. So to create this generalized classifier, we used the sentiment data from the above corpus from different domains. We first tested the performance of sentiment analysis models built using single data source for both in-domain and cross-domain classification. Later, we built sentiment model using data samples from multiple domains and then tested the performance of the models based on their classification. Finally, we compared all the three approaches based on the performance of the models and discussed the best approach for sentiment analysis. Keywords: Sentiment analysis, in-domain, cross-domain, machine learning, reviews, support vector machine, bernoulli naive bayes. 1. Introduction With the rapid increase of textual content on the internet, efficient text processing is very important for various applications. With the advancement of machine learning approaches, the issue of processing can be addressed to a decent level (Hirschberg and Manning, 2015). Automated sentiment analysis is one of the important research topics. For example: Sentiment analysis is very useful in social media monitoring as it allows us to gain an overview of the wider public opinion. Most of the sentiment analysis approaches use supervised machine learning algorithms or expert-defined lexicons. The automated sentiment analysis is a challenging task because of the natural language processing overheads like intentions of the author and the sentiment of the text can change depending on the situation. It becomes a more challenging task for non-English texts because of the unavailability of annotated data sets. In this paper, we explain how to create suitable corpora for supervised machine learning approaches i.e how to extract data from various sources and annotate them using an appropriate …

Content truncated to 500 words