Clustering Word Embeddings with Self-Organizing Maps. Application on LaRoSeDa - A Large Romanian Sentiment Data Set

Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics, pages 949–956 April 19 - 23, 2021. ©2021 Association for Computational Linguistics 949 Clustering Word Embeddings with Self-Organizing Maps. Application on LaRoSeDa – A Large Romanian Sentiment Data Set Anca Maria Tache1,∗, Mihaela G˘aman1,‡, Radu Tudor Ionescu1,2,⋄ 1Department of Computer Science, 2Romanian Young Academy University of Bucharest 14 Academiei, Bucharest, Romania ∗anca.taches@gmail.com ‡mp.gaman@gmail.com ⋄raducu.ionescu@gmail.com Abstract Romanian is one of the understudied lan- guages in computational linguistics, with few resources available for the development of nat- ural language processing tools. In this paper, we introduce LaRoSeDa, a Large Romanian Sentiment Data Set1, which is composed of 15,000 positive and negative reviews collected from one of the largest Romanian e-commerce platforms. We employ two sentiment clas- sification methods as baselines for our new data set, one based on low-level features (character n-grams) and one based on high- level features (bag-of-word-embeddings gen- erated by clustering word embeddings with k- means). As an additional contribution, we re- place the k-means clustering algorithm with self-organizing maps (SOMs), obtaining better results because the generated clusters of word embeddings are closer to the Zipf’s law distri- bution, which is known to govern natural lan- guage. We also demonstrate the generaliza- tion capacity of using SOMs for the cluster- ing of word embeddings on another recently- introduced Romanian data set, for text catego- rization by topic. 1 Introduction Perhaps one of the most studied tasks in com- putational linguistics is sentiment classification, a.k.a. opinion mining or polarity classification. The task has been studied across several languages, the most popular being English (Blitzer et al., 2007; Dos Santos and Gatti, 2014; Fu et al., 2018; Gim´enez-P´erez et al., 2017; Huang et al., 2017; Ionescu and Butnaru, 2019; Kim, 2014; Maas et al., 2011; Pang and Lee, 2005; Shen et al., 2018; Socher et al., 2013), Chinese (Peng et al., 2017; Wan, 2008; Zagibalov and Carroll, 2008; Zhai et al., 2011; Zhang and He, 2013; Zhang 1https://github.com/ancatache/LaRoSeDa et al., 2008), Arabic (Al-Ayyoub et al., 2018; El- sahar and El-Beltagy, 2015; Dahou et al., 2016; Nabil et al., 2013, 2015) or Spanish (Brooke et al., 2009; Molina-Gonz´alez et al., 2015; Navas-Loro and Rodr´ıguez-Doncel, 2019; Vilares et al., 2015; Zafra et al., 2017). However, studying this task for under-studied languages, e.g. Romanian, is diffi- cult without access to large data sets. We hereby in- troduce LaRoSeDa, a Large Romanian Sentiment Data Set, which is freely available for download at https://github.com/ancatache/LaRoSeDa for non- commercial use. With a total of 15,000 positive and negative reviews collected from one of the most popular Romanian e-commerce websites, to our knowledge, LaRoSeDa is the largest data set for Romanian polarity classification. We experiment with two baseline methods on our novel data set. The first baseline employs string kernels, an approach based on low-level features (character n-grams), that was found to work well for sentiment analysis across multiple languages, e.g. English (Gim´enez-P´erez et al., 2017; Ionescu and Butnaru, 2018), Chinese (Zhang et al., …

Content truncated to 500 words