Hybrid Deep Belief Networks for Semi-supervised Sentiment Classification

Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, pages 1341–1349, Dublin, Ireland, August 23-29 2014. Hybrid Deep Belief Networks for Semi-supervised Sentiment Classification Shusen Zhou⋆ Qingcai Chen† Xiaolong Wang† Xiaoling Li⋆ ⋆ School of Information and Electrical Engineering, Ludong University, Yantai 264025, China. † Shenzhen Graduate School, Harbin Institute of Technology, Shenzhen 518055, China. zhoushusen@gmail.com, qingcai.chen@hitsz.edu.cn wangxl@insun.hit.edu.cn, appleling@live.cn Abstract In this paper, we develop a novel semi-supervised learning algorithm called hybrid deep be- lief networks (HDBN), to address the semi-supervised sentiment classification problem with deep learning. First, we construct the previous several hidden layers using restricted Boltzmann machines (RBM), which can reduce the dimension and abstract the information of the reviews quickly. Second, we construct the following hidden layers using convolutional restricted Boltz- mann machines (CRBM), which can abstract the information of reviews effectively. Third, the constructed deep architecture is fine-tuned by gradient-descent based supervised learning with an exponential loss function. We did several experiments on five sentiment classification datasets, and show that HDBN is competitive with previous semi-supervised learning algorithm. Ex- periments are also conducted to verify the effectiveness of our proposed method with different number of unlabeled reviews. 1 Introduction Recently, more and more people write reviews and share opinions on the World Wide Web, which present a wealth of information on products and services (Liu et al., 2010). These reviews will not only help other users make better judgements but they are also useful resources for manufacturers of products to keep track and manage customer opinions (Wei and Gulla, 2010). However, there are large amount of reviews for every topic, it is difficult for a user to manually learn the opinions of an interesting topic. Sentiment classification, which aims to classify a text according to the expressed sentimental polarities of opinions such as ’positive’ or ’negtive’, ’thumb up’ or ’thumb down’, ’favorable’ or ’unfavorable’ (Li et al., 2010), can facilitate the investigation of corresponding products or services. In order to learn a good text classifier, a large number of labeled reviews are often needed for training (Zhen and Yeung, 2010). However, labeling reviews is often difficult, expensive or time consuming (Chapelle et al., 2006). On the other hand, it is much easier to obtain a large number of unlabeled reviews, such as the growing availability and popularity of online review sites and personal blogs (Pang and Lee, 2008). In recent years, a new approach called semi-supervised learning, which uses large amount of unlabeled data together with labeled data to build better learners (Zhu, 2007), has been developed in the machine learning community. There are several works have been done in semi-supervised learning for sentiment classification, and get competitive performance (Li et al., 2010; Dasgupta and Ng, 2009; Zhou et al., 2010). However, most of the existing semi-supervised learning methods are still far from satisfactory. As shown by several re- searchers (Salakhutdinov and Hinton, 2007; Hinton et al., 2006), deep architecture, which composed of multiple levels of non-linear operations, is expected to perform …

Content truncated to 500 words