Variational Weakly Supervised Sentiment Analysis with Posterior Regularization

Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics, pages 3259–3268 April 19 - 23, 2021. ©2021 Association for Computational Linguistics 3259 Variational Weakly Supervised Sentiment Analysis with Posterior Regularization Ziqian Zeng, Yangqiu Song Department of CSE, HKUST, Hong Kong, China {zzengae, yqsong}@cse.ust.hk Abstract Sentiment analysis is an important task in natu- ral language processing (NLP). Most of exist- ing state-of-the-art methods are under the su- pervised learning paradigm. However, human annotations can be scarce. Thus, we should leverage more weak supervision for sentiment analysis. In this paper, we propose a posterior regularization framework for the variational approach to the weakly supervised sentiment analysis to better control the posterior distri- bution of the label assignment. The intuition behind the posterior regularization is that if extracted opinion words from two documents are semantically similar, the posterior distribu- tions of two documents should be similar. Our experimental results show that the posterior regularization can improve the original varia- tional approach to the weakly supervised sen- timent analysis and the performance is more stable with smaller prediction variance. 1 Introduction Sentiment analysis is a task of identifying the senti- ment polarity expressed in textual data (Liu, 2012). Most state-of-the-art sentiment analysis methods in the literature are supervised methods which re- quire many labeled training data. However, human annotations in the real world are scarce. While we assume there is abundant annotated data to train more and more complex models, there is still a need to consider weakly supervised methods that require less human annotation. One way to perform weakly supervised senti- ment analysis is using a predefinited lexicon (Tur- ney, 2002; Taboada et al., 2011). A lexicon consists of many opinion words. For each opinion word, its polarity (positive or negative) and strength (the degree to which the opinion word is positive or negative) are annotated by domain experts. lexicon- based weakly supervised methods perform a dictio- nary lookup and assign a polarity according to all opinion words extracted from a document. A good lexicon requires high precision and high coverage, which needs a lot of human effort. Another way to do weakly supervised sentiment analysis is using limited keywords (Meng et al., 2018; Zeng et al., 2019). Compared with lexicon- based methods, user-provided keywords require less human effort. Among keyword-based meth- ods, there are two directions. First, (Meng et al., 2018) leveraged limited keywords to expand more keywords and generate pseudo-labeled data, and then performed self-training on real unlabeled data for model refinement. Possible improvements of this direction include investigating more advanced keywords expansion techniques to generate bet- ter pseudo-labeled samples (Miller et al., 2012) and developing more advanced self-training algo- rithms(Coden et al., 2014). Second, the Variational Weakly Supervised (VWS) sentiment analysis (Zeng et al., 2019) used target-opinion word pairs as supervision signal. Its objective function is to predict an opinion word given a target word. For example, in a sentence “the room is big,” “room” is a target word and “big” is an opinion …

Content truncated to 500 words