Mere Contrastive Learning for Cross-Domain Sentiment Analysis

Proceedings of the 29th International Conference on Computational Linguistics, pages 7099–7111 October 12–17, 2022. 7099 Mere Contrastive Learning for Cross-Domain Sentiment Analysis Yun Luo 1,2, Fang Guo 2, Zihan Liu 2, Yue Zhang 2,3 1 School of Computer Science And Technology, Zhejiang University, Hangzhou 310024, P.R. China. 2 School of Engineering, Westlake University, Hangzhou, China. 3 Institute of Advanced Technology, Westlake Institute for Advanced Study, Hangzhou, China. {luoyun, guofang, liuzihan, zhangyue}@westlake.edu.cn Abstract Cross-domain sentiment analysis aims to pre- dict the sentiment of texts in the target domain using the model trained on the source domain to cope with the scarcity of labeled data. Pre- vious studies are mostly cross-entropy-based methods for the task, which suffer from insta- bility and poor generalization. In this paper, we explore contrastive learning on the cross- domain sentiment analysis task. We propose a modified contrastive objective with in-batch negative samples so that the sentence repre- sentations from the same class can be pushed close while those from the different classes be- come further apart in the latent space. Experi- ments on two widely used datasets show that our model can achieve state-of-the-art perfor- mance in both cross-domain and multi-domain sentiment analysis tasks. Meanwhile, visualiza- tions demonstrate the effectiveness of transfer- ring knowledge learned in the source domain to the target domain and the adversarial test verifies the robustness of our model. 1 Introduction Sentiment classification (Liu, 2012) has been widely studied by both industry and academia (Blitzer et al., 2007; Li et al., 2013; Yu and Jiang, 2016). For example, the sentiment is posi- tive towards the text ‘The book is exactly as pic- tured/described. Cute design and good quality’. Early methods rely on labeled data to train models on a specific domain (e.g. DVD reviews, book re- views, and so on), which are labor-intensive and time-consuming (Socher et al., 2013). To address this issue, cross-domain sentiment analysis attracts increasing attention. Various neural models have been proposed for cross-domain sentiment analysis in recent years (Blitzer et al., 2007; Li et al., 2013; Yu and Jiang, 2016; Zhang et al., 2019; Zhou et al., 2020a). Most methods focus on making the model unable to dis- tinguish the data from which domain by adversarial Contrastive Training Trained Representations KNN Predictor Positive Negative Pull Push KNN BERT Representations Cross Entropy BERT Representations Decision Boundary Softmax Predictor (a) Cross-entropy Based Model (b) Contrastive Learning Based Model Test Figure 1: The architectures for cross-entropy-based model and the contrastive- learning-based model. training or data argumentation, in order to trans- fer knowledge from source domains to target do- mains (Du et al., 2020; Liu et al., 2017; Qu et al., 2019; Yang et al., 2022) and some attempt to learn domain-specific knowledge (Zhou et al., 2020a; Liu et al., 2018; Wang et al., 2019). Pre-trained lan- guage models (Kenton et al., 2019; Radford et al., 2019; Lewis et al., 2020) have achieved stronger performance compared with previous random ini- tialized models such as LSTM (Long Short-term Memory) in cross-domain tasks. The state-of-the- …

Content truncated to 500 words