Adullam at SemEval-2017 Task 4: Sentiment Analyzer Using Lexicon Integrated Convolutional Neural Networks with Attention

Proceedings of the 11th International Workshop on Semantic Evaluations (SemEval-2017), pages 732–736, Vancouver, Canada, August 3 - 4, 2017. c⃝2017 Association for Computational Linguistics Adullam at SemEval-2017 Task 4: Sentiment Analyzer based on Lexicon Integrated Convolutional Neural Networks with Attention Joosung Yoon Korea University Seoul, South Korea xelloss705@gmail.com Kigon Lyu Korea University Seoul, South Korea gon0121@korea.ac.kr Hyeoncheol Kim Korea University Seoul, South Korea hkim64@gmail.com Abstract We propose a sentiment analyzer for the prediction of document-level sentiments of English micro-blog messages from Twit- ter. The proposed method is based on lexi- con integrated convolutional neural net- works with attention (LCA). Its perfor- mance was evaluated using the datasets provided by SemEval competition (Task 4). The proposed sentiment analyzer ob- tained an average F1 of 55.2%, an average recall of 58.9% and an accuracy of 61.4%. 1 Introduction Sentiment analysis is necessary to interpret the vast number of online opinions on social media platforms such as Twitter. This will allow gov- ernments and corporations to manage public rela- tions and policies effectively. Existing sentiment analyzers are based on naïve bayes, SVM, RNN (Irsoy, 2014) and in particular convolutional neu- ral networks (CNNs) (Kim, 2014). In order to improve on existing CNN based sen- timent analyzer, lexicon embedding and attention embedding were integrated into the proposed sen- timent analyzer. Lexicon embedding allows ex- traction of sentimental score for each word and at- tention embedding enables the global view of the sentence. The proposed LCA was both trained and evalu- ated using corpus from Twitter 2013 to 2016 pro- vided by the SemEval-2017. Figure 1 shows the overview of the proposed sentiment analyzer. It consists of embedding, CNNs, concatenation, ful- ly connected and softmax layer. 2 Input Features & Architecture The proposed LCA consists of three input features (i) Word embeddings (ii) Lexicon embeddings (iii) Attention embeddings. Figure 1: Architecture of proposed LCA. 732 Word embeddings are trained by implementation of word2vec using skip-gram (Mikolov, 2014) and negative sampling. The word embeddings are trained using an unlabeled corpus of 1.6M tweets from Sentiment 140 dataset with different dimen- sions (50, 100, 200, 400). The dimensions of word embeddings are 𝑑𝑑 and the number of words in a document is 𝑛𝑛 Lexicon embeddings are considered because they are useful features. Lexicon embeddings consist of set of words each paired with a score ranging from -1 to +1. Where a score of -1 represents a negative sentiment and +1, a positive sentiment. The lexicon document corresponding to each word is 𝑠𝑠𝑙𝑙 ∈ ℝ𝑛𝑛×𝑒𝑒, where 𝑒𝑒 is the dimension of lexicon embeddings and it is set by the number of lexicon corpus. Attention embeddings are important for Deep Learning in terms of performance and explanation of models (Kelvin, 2015). CNN uses several fil- ters which have length 𝑙𝑙. It considers 𝑙𝑙-gram fea- tures, but it only takes local views into account not considering the global view of sentence. Sen- timent analysis must consider transitional cases such as negation. While attention embeddings can capture keywords to improve sentiment …

Content truncated to 500 words