Sentence Modeling with Deep Neural Architecture using Lexicon and Character Attention Mechanism for Sentiment Classification

Proceedings of the The 8th International Joint Conference on Natural Language Processing, pages 536–544, Taipei, Taiwan, November 27 – December 1, 2017 c⃝2017 AFNLP Sentence Modeling with Deep Neural Architecture using Lexicon and Character Attention Mechanism for Sentiment Classification Huy Thanh Nguyen and Minh Le Nguyen Japan Advanced Institute of Science and Technology Ishikawa, Japan {huy.nguyen, nguyenml}@jaist.ac.jp Abstract Tweet-level sentiment classification in Twitter social networking has many chal- lenges: exploiting syntax, semantic, sen- timent and context in tweets. To ad- dress these problems, we propose a novel approach to sentiment analysis that uses lexicon features for building lex- icon embeddings (LexW2Vs) and gen- erates character attention vectors (Char- AVs) by using a Deep Convolutional Neu- ral Network (DeepCNN). Our approach integrates LexW2Vs and CharAVs with continuous word embeddings (Continu- ousW2Vs) and dependency-based word embeddings (DependencyW2Vs) simulta- neously in order to increase information for each word into a Bidirectional Contex- tual Gated Recurrent Neural Network (Bi- CGRNN). We evaluate our model on two Twitter sentiment classification datasets. Experimental results show that our model can improve the classification accuracy of sentence-level sentiment analysis in Twit- ter social networking. 1 Introduction Tweet-level sentiment classification is a funda- mental task of sentiment analysis in Twitter so- cial networking and is essential to understand user generated contents in social networking. Twit- ter sentiment classification have intensively re- searched in recent years (Go et al., 2009) (Nakov et al., 2016). There are many works related to deep learning methods involved learning word repre- sentation (Socher et al., 2011). Word represen- tation is central to deep learning and essential feature extractor that encode different features of words in their dimensions. The combination of word representation and deep learning achieved impressive results because word embeddings en- able efficient computation of word similarities through low-dimensional matrix operations (Kim, 2014). In addition, deep learning models achieved remarkable performance. Some researchers use Convolution Neural Network (CNN) for sentiment classification. CNN utilizes convolution filters ap- plied to local features. CNN models has been shown to be effective for NLP. For example, the model of (Dos Santos and Gatti, 2014) used CNN to form a sentence-level representation for sen- timent classification. In addition, Bidirectional Gated Recurrent Neural Network (Bi-GRNN) is another deep learning model that has achieved an excellent result for sentiment analysis and other traditional tasks (Chung et al., 2014). Inspired by the models above, the goal of this research is to build a model for exploiting syn- tax, semantic, sentiment and context of tweets by constructing four kinds of embeddings: Char- AVs, LexW2Vs, ContinuousW2Vs and Depen- dencyW2Vs. On the other hand, we modify Bi- GRNN of (Chung et al., 2014) into Bi-CGRNN to take word embeddings in order to produce a sentence-wide representation from sentence com- positions. Our paper makes the following contri- butions: • We construct a tweet processor which stan- dardizes tweets by using pre-processing steps and a semantic rule-based approach. We con- struct four kinds of embeddings: CharAvs, LexW2Vs, ContinuousW2Vs and Dependen- cyW2Vs. A DeepCNN is used for …

Content truncated to 500 words