Vision Language Pre-training by Contrastive Learning with Cross-Modal Similarity Regulation

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 1456–1470 July 9-14, 2023 ©2023 Association for Computational Linguistics Cross-Domain Data Augmentation with Domain-Adaptive Language Modeling for Aspect-Based Sentiment Analysis Jianfei Yu∗, Qiankun Zhao∗ and Rui Xia† School of Computer Science and Engineering, Nanjing University of Science and Technology, China {jfyu, kkzhao, rxia}@njust.edu.cn Abstract Cross-domain Aspect-Based Sentiment Analy- sis (ABSA) aims to leverage the useful knowl- edge from a source domain to identify aspect- sentiment pairs in sentences from a target do- main. To tackle the task, several recent works explore a new unsupervised domain adapta- tion framework, i.e., Cross-Domain Data Aug- mentation (CDDA), aiming to directly generate much labeled target-domain data based on the labeled source-domain data. However, these CDDA methods still suffer from several is- sues: 1) preserving many source-specific at- tributes such as syntactic structures; 2) lack of fluency and coherence; 3) limiting the diver- sity of generated data. To address these issues, we propose a new cross-domain Data Augmen- tation approach based on Domain-Adaptive Language Modeling named DA2LM, which contains three stages: 1) assigning pseudo la- bels to unlabeled target-domain data; 2) unify- ing the process of token generation and label- ing with a Domain-Adaptive Language Model (DALM) to learn the shared context and an- notation across domains; 3) using the trained DALM to generate labeled target-domain data. Experiments show that DA2LM consistently outperforms previous feature adaptation and CDDA methods on both ABSA and Aspect Ex- traction tasks. The source code is publicly re- leased at https://github.com/NUSTM/DALM. 1 Introduction As an important task in sentiment analysis, Aspect- Based Sentiment Analysis (ABSA) aims to extract aspect terms from sentences and predict the senti- ment polarity towards each aspect term (Liu, 2012; Pontiki et al., 2016). For example, given a sentence “The screen is broken", the aspect term is screen and its sentiment polarity is Negative. With the ad- vancements of deep learning techniques, a myriad of neural approaches have been proposed for ABSA ∗Equal contribution. † Corresponding author. Decoder Encoder The [MASK] was pretty great . O B-ASP O O B-OP O pizza The laptop was pretty great . O B-ASP O O B-OP O b) Seq2Seq-based Cross-Domain Data Augmentation Masked source data (Restaurant): Generated target data (Laptop): MLMs Order the [MASK] [MASK] positive, it's fantastic. panang duck Order the glass touchpad positive, it's fantastic. a) MLM-based Cross-Domain Data Augmentation Masked source data (Restaurant): Generated target data (Laptop): Decoder Labeled source data (Restaurant) Pseudo-labeled data (Laptop) Toshiba will not acknowledge this laptop negative as a repair. ...... ...... Train Generated target data (Laptop): c) DALM-based Cross-Domain Data Augmentation (Ours) Figure 1: Comparison between different Cross-Domain Data Augmentation (CDDA) methods. and achieved promising results on several bench- mark datasets (Li et al., 2019a; He et al., 2019; Chen and Qian, 2020b). However, these methods heavily rely on labeled data with fine-grained anno- tation, which is often time-consuming and expen- sive to obtain for many emerging domains. To alleviate the reliance on …

Content truncated to 500 words