Multi-Channel Lexicon Integrated CNN-BiLSTM Models for Sentiment Analysis

Multi-Channel Lexicon Integrated CNN-BiLSTM Models for Sentiment Analysis Joosung Yoon Korea University Seoul, South Korea xelloss705@gmail.com Hyeoncheol Kim Korea University Seoul, South Korea hkim64@gmail.com Abstract We improved sentiment classifier for predicting document-level sentiments from Twitter by using multi-channel lexicon embedidngs. The core of the architecture is based on CNN- BiLSTM that can capture high level features and long term dependency in documents. We also applied multi-channel method on lexicon to improve lexicon features. The macro- averaged F1 score of our model outperformed other classifiers in this paper by 1-4%. Our model achieved F1 score of 64% in SemEval Task 4 (2013-2016) datasets when multi- channel lexicon embedding was applied with 100 dimensions of word embedding. Keywords: Deep Learning, Lexicon, Multi-Channel, CNN-BiLSTM, Sentiment analysis 1. Introduction Sentiment analysis, known as opinion mining is a task of natural language processing (NLP) aimed to identify sentiment polarities expressed in documents. Numerous amounts of opinioned texts are created on social media every day. For instance, Twitter users generate over 500 million tweets daily. It is important to analyze these opinioned texts because they give useful information such as response for specific product, opinion for candidates and etc. However, in sentiment analysis, sarcasm is difficult to distinguish. Usually, sentiment classifier can identify polarity better in the case of clear expression than in the case of sarcasm. Contextualization and informal language in social media are additional complicating factors to sentiment classifier (Deriu et al, 2017). To solve this problem, our approach focuses on high level features of document extracted by 244 The 2017 Conference on Computational Linguistics and Speech Processing ROCLING 2017, pp. 244-253  The Association for Computational Linguistics and Chinese Language Processing CNN and the context considered by BiLSTM that capture long term dependency which helps to understand the context. Therefore, we propose a Multi-Channel Lexicon Integrated CNN- BiLSTM (MCLICB) model for sentiment analysis. Our contributions are: (i) To improve performance of sentiment classifier (ii) To introduce multi-channel lexicon embeddings and analyze influence for sentiment analysis. 2. Related Works The first success of sentiment analysis based on convolutional neural networks (CNN) was triggered by text classification (Kim, 2014). This work provided simple and effective architecture for text classification. Convolutional layer can extract local n-gram features. After this research, various modified models based on CNN have been proposed. One of the modified models is lexicon integrated CNN model with attention (Shin and Lee and Choi, 2016). In the traditional setting, where statistical models are based on hand-crafted features, lexicon is a useful feature, consisting of words and their sentiment scores. CNN architecture of Shin showed that lexicon embedding still can be a useful feature for sentiment analysis. CNN based methods have been successful in many NLP tasks. However, it has limitations in respect of long term dependency. In contrast, Long Short-Term Memory (LSTM) (Hochreiter et al., 1997; Tai et al., 2015) can capture semantic information with long term dependency. In order to consider local n-gram features and long term dependency, various models which combined both …

Content truncated to 500 words