SPACL: Shared-Private Architecture based on Contrastive Learning for Multi-domain Text Classification

SPACL: Shared-Private Architecture based on Contrastive Learning for Multi-domain Text Classification Guoding Xiong1, Yongmei Zhou1,2∗, Deheng Wang1, Zhouhao Ouyang3 1School of Cyber Security, Guangdong University of Foreign Studies, Guangzhou 510006, China 2Guangzhou Key Laboratory of Multilingual Intelligent Processing, Guangdong University of Foreign Studies, Guangzhou 510006, China 3 School of Computing, University of Leeds, Wood-house Lane, Leeds, West Yorkshire, LS2 9JT, United Kingdom 1184467802@qq.com,yongmeizhou@gdufs.edu.cn 1148684516@qq.com,tal-darim@foxmail.com Abstract With the development of deep learning in recent years, text classification research has achieved remarkable results. However, text classification task often requires a large amount of annotated data, and data in different fields often force the model to learn different knowledge. It is often dif- ficult for models to distinguish data labeled in different domains. Sometimes data from different domains can even damage the classification ability of the model and reduce the overall perfor- mance of the model. To address these issues, we propose a shared-private architecture based on contrastive learning for multi-domain text classification which can improve both the accuracy and robustness of classifiers. Extensive experiments are conducted on two public datasets. The results of experiments show that the our approach achieves the state-of-the-art performance in multi-domain text classification. 1 Introduction Text classification is one of the most basic tasks among the many tasks of Natural Langugae Process- ing(NLP). In recent years, the research work of text classification has produced a large number of ap- plications and achieved remarkable results. With the continuous release of a large number of pretrained language models in recent years, such as BERT (Devlin et al., 2018), ALBERT (Lan et al., 2019), RoBERTa (Liu et al., 2019) and other pretrained models, text classification problems have been able to achieve good results on the basis of neural network and pretrained models. However, most text classi- fication problems are highly domain-dependent in that the meaning of the same word may transform in different domains. For example, the word apple expresses the fruit in kitchen review (e.g., I have shifted to an apple for lunch ), while in electronics review, it means a brand of electronic products(e.g., I can’t understand how apple sell so much ipod video). A common strategy, training multiple classifiers for dif- ferent domains, is used to solve above problems. However, text data in reality often have characteristics of multiple domains and the cost of labeling a large number of multi-domain data is too high. Therefore, it is very important and practical meaningful to improve the accuracy of text classification in multiple re- lated domains. Multi-domain text classification(MDTC) (Li and Zong, 2008) is proposed to solve above problems, it aims to utilize textual information in different domains to improve the performance of model architecture, but there is no need to train a separate classifier for each domain. In recent years, deep learn- ing has been widely used in MDTC problems, and has achieved excellent results (Wu and Huang, 2015; Wu and Guo, 2020). The method used in most studies is shared-private architecture. Private modules are used to capture domain-specific …

Content truncated to 500 words