RED: A Novel Dataset for Romanian Emotion Detection from Tweets

Proceedings of Recent Advances in Natural Language Processing, pages 291–300 Sep 1–3, 2021. https://doi.org/10.26615/978-954-452-072-4_034 291 RED: A Novel Dataset for Romanian Emotion Detection from Tweets Alexandra Ciobotaru Advanced Technologies Institute Bucharest, Romania alexandra.ciobotaru@dcti.ro Liviu P. Dinu University of Bucharest Bucharest, Romania liviu.p.dinu@gmail.com Abstract In Romanian language there are some re- sources for automatic text comprehension, but for Emotion Detection, not lexicon-based, there are none. To cover this gap, we extracted data from Twitter and created the first dataset containing tweets annotated with five types of emotions: joy, fear, sadness, anger and neu- tral, with the intent of being used for opinion mining and analysis tasks. In this article we present some features of our novel dataset, and create a benchmark to achieve the first super- vised machine learning model for automatic Emotion Detection in Romanian short texts. We investigate the performance of four clas- sical machine learning models: Multinomial Naive Bayes, Logistic Regression, Support Vector Classification and Linear Support Vec- tor Classification. We also investigate more modern approaches like fastText, which makes use of subword information. Lastly, we fine- tune the Romanian BERT for text classifica- tion and our experiments show that the BERT- based model has the best performance for the task of Emotion Detection from Romanian tweets. Keywords: Emotion Detection, Twitter, Roma- nian, Supervised Machine Learning 1 Introduction Romanian language is a very little explored lan- guage in terms of Natural Language Process- ing (NLP) and Natural Language Understanding (NLU), but social media is full of data and people sharing their opinions about diverse topics, thus some opinion mining tools would be very useful. Opinion mining is defined in Ravi (2015) as “the task of detecting, extracting and classifying opin- ions, sentiments and attitudes concerning different topics, as expressed in textual input”. In NLP tasks, there are two major subcategories of opinion min- ing: Sentiment Analysis (SA) and Emotion Detec- tion (ED). The main difference between these two is that sentiment analysis categorizes opinions be- tween positive or negative, while emotion detection aims to extract the specific emotion a text gives to the reader. English language is rich in NLP resources - we can see not only that there are many manually la- belled datasets, with a large variety of emotions, but there also exist datasets with tweets labeled by emotion intensity (for example, WASSA dataset by Mohammad and Bravo-Marquez (2017) which was first presented at the 2017 Shared Task on Emotion Intensity). Detecting emotions can be a subjective task, even psychologists didn’t agree upon what the main emotions of a human being are. Plutchik (1973) identified eight primary emotions: ecstasy, admiration, terror, amazement, grief, loathing, rage and vigilance, while Ekman (2005) said that the primary emotions are: joy, sadness, fear, disgust, surprise and anger. Inspired by the work of Mo- hammad and Bravo-Marquez (2017), we decided to collect tweets from Twitter and manually an- notate them for four emotions: fear, anger, joy, sadness, and also add a neutral class. We show the quality of the overall dataset …

Content truncated to 500 words