Asymmetric Mutual Learning for Multi-source Unsupervised Sentiment Adaptation with Dynamic Feature Network

Proceedings of the 29th International Conference on Computational Linguistics, pages 6934–6943 October 12–17, 2022. 6934 Asymmetric Mutual Learning for Multi-source Unsupervised Sentiment Adaptation with Dynamic Feature Network Rui Li*, Cheng Liu, Dazhi Jiang Department of Computer Science, Shantou University {ruili, cliu, dzjiang}@stu.edu.cn Abstract Recently, fine-tuning the pre-trained language model (PrLM) on labeled sentiment datasets demonstrates impressive performance. How- ever, collecting labeled sentiment dataset is time-consuming, and fine-tuning the whole PrLM brings about much computation cost. To this end, we focus on multi-source unsuper- vised sentiment adaptation problem with the pre-trained features, which is more practical and challenging. We first design a dynamic feature network to fully exploit the extracted pre-trained features for efficient domain adapta- tion. Meanwhile, with the difference of the tra- ditional source-target domain alignment meth- ods, we propose a novel asymmetric mutual learning strategy, which can robustly estimate the pseudo-labels of the target domain with the knowledge from all the other source mod- els. Experiments on multiple sentiment bench- marks show that our method outperforms the recent state-of-the-art approaches, and we also conduct extensive ablation studies to verify the effectiveness of each the proposed module. 1 Introduction Sentiment classification (Cambria et al., 2020) aims to predict the sentiment label for each textual data automatically (Susanto et al., 2022), which is one of the most popular natural language pro- cessing (NLP) tasks with many important applica- tions, such as social media monitoring (Ortigosa et al., 2014), market research (Jabbar et al., 2019), conversation sentiment detection (Tu et al., 2022), etc. Very recently, the pre-trained language models (PrLMs), e.g., BERT (Devlin et al., 2019), have demonstrated significant improvements on wide- range of NLP tasks, including the sentiment clas- sification. This framework includes two steps: the transformer-based (Vaswani et al., 2017) model is first pre-trained on large unlabeled corpus, and then fine-turned on the labeled datasets for the downstream tasks. However, as illustrated in Fig- Figure 1: Illustration of domain-shift for sentiment clas- sification. Top: different domain reviews have different subject words (marked with blue) with different senti- ment descriptions (marked with underlines). Bottom: the different data distributions will lead to performance degradation. ure 1(Top), different domain texts often contain dif- ferent subject words, and have different sentiment descriptions, which lead to decreased performance induced by domain-shift (Pan and Yang, 2010)(Fig- ure 1(Bottom)). Unsupervised domain adaptation (UDA) is an- other hot research topic in machine learning to address the domain-shift. It aims to transfer the knowledge from the source domain to the target domain with the labeled source data and unlabeled target data. This is often achieved by minimiz- ing a specific distance between the source and tar- get domains to learn the shared domain-invariant features. For example, Guo et al. (2020) exploit several distance metrics (e.g., Maximum Mean Dis- crepancy (MMD) Gretton et al., 2012, Correlation Alignment (CORAL) Sun et al., 2016) for domain adaptation in the context of text classification tasks. While, Li et al. (2017) use a domain classifier to obtain domain-invariant sentiment features via ad- versarial …

Content truncated to 500 words