Annotated Corpus for Sentiment Analysis in Odia Language

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 2788–2795 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 2788 Annotated Corpus for Sentiment Analysis in Odia Language Gaurav Mohanty, Pruthwik Mishra, Radhika Mamidi Language Technologies Research Center International Institute of Information Technology-Hyderabad Telangana, India {gaurav.mohanty,pruthwik.mishra}@research.iiit.ac.in, radhika.mamidi@iiit.ac.in Abstract Given the lack of an annotated corpus of non-traditional Odia literature which serves as the standard when it comes sentiment analysis, we have created an annotated corpus of Odia sentences and made it publicly available to promote research in the field. Secondly, in order to test the usability of currently available Odia sentiment lexicon, we experimented with various classifiers by training and testing on the sentiment annotated corpus while using identified affective words from the same as features. Annotation and classification are done at sentence level as the usage of sentiment lexicon is best suited to sentiment analysis at this level. The created corpus contains 2045 Odia sentences from news domain annotated with sentiment labels using a well-defined annotation scheme. An inter-annotator agreement score of 0.79 is reported for the corpus. Keywords: Odia Corpus, Sentiment Analysis, Language Resource, Resource Creation 1. Introduction Most of the research in sentiment analysis is focused on genres such as news data, customer reviews and tweets. Sentiment annotated corpus is useful to build models for the task of sentiment analysis. For these genres, annotation usually takes place at sentence and phrase level. Odia1, being a resource-poor language, does not have such an annotated corpus available for public use. However, we have attempted to create an annotated corpus for Odia poetry, previously in liter- ature (Mohanty et al., 2018). In this paper, we create a sentiment annotated cor- pus of Odia sentences in News domain. News data may have opinionated references along with factual data. Hence at a sentence level these can be classified into positive, negative and neutral categories. More- over, sentence level sentiment analysis provides room for usage of a sentiment lexicon for identifying affective words. We built an Odia sentiment lexicon for the task of sentiment classification previously (Mohanty et al., 2017). This lexicon has been built by using resources available for three other Indian languages: Bengali, Telugu, Tamil (Das and Bandyopadhyay, 2010) which are similar to Odia. IndoWordNet (Bhattacharyya, 2010) was used for establishing language pairs between Odia and each of the three aforementioned Indian lan- guages. Classification performance using the Odia sen- timent lexicon should provide valuable insight on the usability of this sentiment lexicon. We have created an annotated corpus of Odia sentences from the abundantly available data in news domain for the language. This has further been made publicly available to promote research in the field. Secondly, in order to test the usability the already present Odia sentiment lexicon, we experimented with various clas- sifiers by training and testing on the sentiment anno- 1https://en.wikipedia.org/wiki/Odia_language tated corpus while using identified affective words from the same as features. Annotation and classification are done at sentence …

Content truncated to 500 words