BLCUFIGHT at SemEval-2021 Task 10: Novel Unsupervised Frameworks For Source-Free Domain Adaptation

Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), pages 357–363 Bangkok, Thailand (online), August 5–6, 2021. ©2021 Association for Computational Linguistics 357 BLCUFIGHT at SemEval-2021 Task 10: Novel Unsupervised Frameworks For Source-Free Domain Adaptation Weikang Wang, Yi Wu, Yixiang Liu, Pengyuan Liu Beijing Language and Culture University, Beijing, China {978955719wwk, wyclover51}@gmail.com lyx blcu@163.com, liupengyuan@pku.edu.cn Abstract Domain adaptation assumes that samples from source and target domains are freely accessi- ble during a training phase. However, such as- sumption is rarely plausible in the real-world and may causes data-privacy issues, especially when the label of the source domain can be a sensitive attribute as an identifier. SemEval- 2021 task 10 focuses on these issues. We par- ticipate in the task and propose novel frame- works based on self-training method. In our systems, two different frameworks are de- signed to solve text classification and sequence labeling. These approaches are tested to be ef- fective which ranks the third among all system in subtask A, and ranks the first among all sys- tem in subtask B. 1 Introduction Deep neural networks have achieved remarkable success in a variety of applications across different fields while with huge expense of laborious large- scale training data annotation. To avoid expensive data labeling, domain adaptation(DA) methods was proposed to fully utilize previously labeled datasets and unlabeled data on hand in a transductive man- ner, which obtained promising results in sentiment analysis, part-of-speech tagging, machine transla- tion, etc. (Glorot et al., 2011; Yang and Eisenstein, 2014; Chu and Wang, 2018) Unsupervised Domain Adaptation(UDA) aims to reduce the domain shift between labeled and un- labeled target domains. Early works (Blitzer et al., 2006; Pan et al., 2010) learnt domain-invariant fea- tures to link the target domain to the source domain. Along with the growing popularity of deep learning, plenty of works benefited from its powerful repre- sentation learning ability for domain adaptation. Those methods typically minimized the distribu- tion discrepancy between two domains (Plank et al., 2014), or deployed adversarial training (Ganin and Lempitsky, 2015; Bousmalis et al., 2016; Li et al., 2018). However, a crucial requirement in the methodol- ogy of these methods is that all samples from both domains are freely available during the training pro- cess, which is inefficient in data transmission and may violate the data privacy policy. For example, it is not allowed to share tweet texts according to Twitter policies, though tweet IDs can be shared. The situation is even more common in clinical NLP, where patient health information must be protected, and annotations over health text, when released at all, often require the signing of complex data use agreements. SemEval 2021 task 10 focuses on the problem of source-free domain adaptation for semantic pro- cessing. Subtask A of task 10 is negation detection which aims to classify clinical event mentions (e.g., diseases, symptoms, procedures) for whether they are being negated by their context. Traditional sys- tems, such as one of the first algorithms NegEx (Chapman et al., 2001) was based on …

Content truncated to 500 words