BAKSA at SemEval-2020 Task 9: Bolstering CNN with Self-Attention for Sentiment Analysis of Code Mixed Text

Proceedings of the 14th International Workshop on Semantic Evaluation, pages 1221–1226 Barcelona, Spain (Online), December 12, 2020. 1221 BAKSA at SemEval-2020 Task 9: Bolstering CNN with Self-Attention for Sentiment Analysis of Code Mixed Text Ayush Kumar∗ Harsh Agarwal∗ Keshav Bansal∗ Indian Institute of Technology Kanpur (IITK)Ashutosh Modi {ayushk,harshaga,keshavb}@iitk.ac.in ashutoshm@cse.iitk.ac.in Abstract Sentiment Analysis of code-mixed text has diversified applications in opinion mining ranging from tagging user reviews to identifying social or political sentiments of a sub-population. In this paper, we present an ensemble architecture of convolutional neural net (CNN) and self-attention based LSTM for sentiment analysis of code-mixed tweets. While the CNN component helps in the classification of positive and negative tweets, the self-attention based LSTM, helps in the classification of neutral tweets, because of its ability to identify correct sentiment among multiple sentiment bearing units. We achieved F1 scores of 0.707 (ranked 5th) and 0.725 (ranked 13th) on Hindi-English (Hinglish) and Spanish-English (Spanglish) datasets, respectively. The submissions for Hinglish and Spanglish tasks were made under the usernames ayushk and harsh 6 respectively. 1 Introduction The research problem of Sentiment Analysis of Code-Mixed Social Media Text appeared as part of the SemEval Shared Challenge 2020 (Patwa et al., 2020). Mixing languages while writing text, also called code-mixing, is a typical pattern observed in almost all forms of communication, including social media text. We only focus on two popular bilingual code-mixing styles namely Hinglish and Spanglish. Sentiment Analysis is a term broadly used to classify states of human affection and emotion. Interpreting code-mixed languages is difficult not only because the sentences may not fit a particular language model, but also because mixed text on social-media usually contains tokens such as hashtags, and usernames. In this paper, we present an ensemble of CNN and self-attention based LSTM, utilizing the XLM-R embeddings (Conneau et al., 2019). While CNNs have been used for sentiment analysis before (Wang et al., 2016; Yoon and Kim, 2017), none of the previous works have used a self-attention based LSTM along with it. We found that while the CNN component worked well for positive and negative tweets, the self-attention component worked better for neutral tweets, necessitating an ensemble of the two. The implementation of our system is made available via Github1. 2 Related Work Performing standard NLP tasks on code-mixed data has presented significant challenges. Vyas et al. (2014) attempted to find methods for POS tagging of code-mixed social media text. Another work by Joshi et al. (2016) used CNNs to learn subword level embeddings and then utilized these embeddings in a BiLSTM network to learn subword level information from social media text. Subword level representations are particularly important while dealing with noisy texts containing mis- spellings and punctuations. However, this work doesn’t capture information about word-level semantics. More recent work by Lal et al. (2019) uses two parallel BiLSTMs, which they call the Collective and Specific Encoder and an additional feature network. This approach combines recurrent neural networks utilizing attention mechanisms, which helps in evaluating the overall sentiment using …

Content truncated to 500 words