YNU-HPCC at SemEval-2021 Task 10: Using a Transformer-based Source-Free Domain Adaptation Model for Semantic Processing

Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), pages 1289–1294 Bangkok, Thailand (online), August 5–6, 2021. ©2021 Association for Computational Linguistics 1289 YNU-HPCC at SemEval-2021 Task 10: Using a Transformer-based Source-Free Domain Adaptation Model for Semantic Processing Zhewen Yu, Jin Wang and Xuejie Zhang School of Information Science and Engineering Yunnan University Kunming, China zhwyu@mail.ynu.edu.cn, {wangjin, xjzhang}@ynu.edu.cn Abstract Data sharing restrictions are common in natural language processing datasets. The aim of this study is to develop a model that is trained in a source domain to make predictions for a target domain with respect to domain data. To address this problem, the organizers provided models that fine-tuned a large number of source domain data on pre-trained models and dev data for partic- ipants. However, source domain data were not distributed. This paper describes the model provided for the name entity recog- nition task and ways to develop the model. Because little data are provided, pre-trained models are suitable for solving cross-domain tasks. The models fine-tuned by a large number of other domains could be effective in the new domain because the task did not change. The code of this paper is available at https://github.com/windforfurture/ SemEval-2021-Task10. 1 Introduction Data sharing constraints are common in natural language processing (NLP) datasets. For exam- ple, Twitter policies prohibit the sharing of tweet text, although tweet IDs may be shared. In clini- cal NLP, the situation is even more prevalent be- cause information on patients’ health must be pro- tected. Obtaining annotations about health texts often requires the signing of complex data usage agreements. During the competition, the organizers provided models which fine-tuned on the annotated source domain data, while the source domain data could not be distributed. The organizers also pro- vided labeled data as dev data and unlabeled target domain data as test data. There were two sub-tasks for SemEval Task 10: 1) to classify clinical event mentions (e.g., dis- eases, symptoms, procedures, etc.) for whether they are being negated by their context. This is a text-classification task (Yuan et al., 2020). 2) to find time expressions (Laparra et al., 2018) in text, this is a sequence-tagging task. SemEval Task 10 was different from traditional NLP tasks, where training and testing were in the same domain. Pre- dictions could be out of control due to the different target domain. The domain of the dev data is re- lated to the source and target domains, therefore the model can be developed. In SemEval Task 10, it is necessary to develop an existing model along with training the model with labeled data or unlabeled data. Furthermore, the model can be developed by fine-tuning it in different ways. For certain reasons, only the results for the sec- ond subtask were submitted, that is, time expres- sion recognition, which can be considered as a name entity recognition (NER) task. The partici- pants were asked to find time expressions in the text through the task. This is a sequence-tagging task that uses fine-grained time expression annotations …

Content truncated to 500 words