HCMS at SemEval-2020 Task 9: A Neural Approach to Sentiment Analysis for Code-Mixed Texts

Proceedings of the 14th International Workshop on Semantic Evaluation, pages 1253–1258 Barcelona, Spain (Online), December 12, 2020. 1253 HCMS at SemEval-2020 Task 9: A Neural Approach to Sentiment Analysis for Code-Mixed Texts Aditya Srivastava International Institute of Information Technology, Hyderabad aditya.srivastava @research.iiit.ac.in V. Harsha Vardhan International Institute of Information Technology, Hyderabad harshavardhan.v @research.iiit.ac.in Abstract Problems involving code-mixed language are often plagued by a lack of resources and an absence of materials to perform sophisticated transfer learning with. In this paper we describe our submission to the Sentimix Hindi-English task involving sentiment classification of code-mixed texts, and with an F1 score of 67.1%, we demonstrate that simple convolution and attention may well produce reasonable results. 1 Introduction Sentiment analysis has gained importance in the current world where social media is crucial in gauging public opinion on products, political campaigns, latest trends and more. A large portion of the world’s population is multilingual, and so is the content on social networks. Code-mixing is a natural phenomenon among bilinguals where phrases and words from one language are employed in another. Typically, the underlying grammar of the primary language that is being spoken is kept intact and phrases from another language are embedded into it. India in particular, has 23 officially recognized languages and a majorly bilingual population, requiring robust computational tools to effectively exploit the data it produces. Deep neural networks have had great success in predicting sentiment encapsulated in text - a big part of this success has been their ability to utilize pretrained word embeddings. In code-mixed tweets however, Hindi words are written in Roman script instead of its native Devanagari script, forcing one to use a transliterator in order to take advantage of pretrained aligned word embeddings, like fastText (Bojanowski et al., 2016). These words in Roman script are unnormalized and may have multiple spellings; for example, bahut can be spelt in Roman script as bahut, bohot, bohut etc., and such errors propagate from the transliterator to our downstream task, thereby reducing performance. In this work, we propose a deep convolution network with self-attention which shows promising results, without any pretraining. Convolution neural networks (CNNs) are known to capture local relations (Kim, 2014) or local context, and work here as feature extractors acting as a substitute for handcrafted sentiment features. A self-attention layer is then applied over these features (presenting as vectors), which allow each individual feature to attend to all other features (Bahdanau et al., 2014), providing global context. We call this the Hierarchical Context Modeling System (henceforth referred to as HCMS), due to the hierarchical nature of context extraction performed on both levels. 2 Background Early sentiment analysis systems made use of corpus based statistics (Wiebe and others, 2000), linguistic tools like Wordnet (Miller, 1998) and lexicon based classifiers (Turney and Littman, 2002). More recently, a large portion of work has involved Recurrent Neural Networks (RNNs), Long Short Term Memory Networks (LSTMs) and CNNs at word and character levels (Mulder et al., 2015; Zhang et al., 2015). Usage …

Content truncated to 500 words