Hardness-guided domain adaptation to recognise biomedical named entities under low-resource scenarios

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 4063 - 4071 December 7-11, 2022 ©2022 Association for Computational Linguistics Hardness-guided domain adaptation to recognise biomedical named entities under low-resource scenarios Ngoc Dang Nguyen Monash University dan.nguyen2@monash.edu Lan Du∗ Monash University lan.du@monash.edu Wray Buntine VinUniversity wray.b@vinuni.edu.vn Changyou Chen University at Buffalo changyou@buffalo.edu Richard Beare Monash University richard.beare@monash.edu Abstract Domain adaptation is an effective solution to data scarcity in low-resource scenarios. How- ever, when applied to token-level tasks such as bioNER, domain adaptation methods often suffer from the challenging linguistic charac- teristics that clinical narratives possess, which leads to unsatsifactory performance. In this pa- per, we present a simple yet effective hardness- guided domain adaptation (HGDA) framework for bioNER tasks that can effectively lever- age the domain hardness information to im- prove the adaptability of the learnt model in the low-resource scenarios. Experimental results on biomedical datasets show that our model can achieve significant performance improve- ment over the recently published state-of-the- art (SOTA) MetaNER model. 1 Introduction Named Entity Recognition (NER) is a fundamen- tal NLP task which aims to locate named entity (NE) mentions and classify them into predefined categories such as location, organization, or person. NER usually serves as an important first sub-task for information retrieval (Banerjee et al., 2019), task oriented dialogues (Peng et al., 2020) and other language applications. Consequently, NER has seen significant performance improvements with the recent advances of pre-trained language models (PLMs) (Akbik et al., 2019; Devlin et al., 2019). Unfortunately, a large amount of training data is often essential for these PLMs to excel and except for a few high-resource domains, the majority of domains have limited amount of labeled data. This data-scarcity problem amplifies given the context of biomedical NER (bioNER). Firstly, the annotation process for the biomedical domains is time-consuming and can be extremely expensive. Thus, many biomedical domain corpora, especially for those privately developed, are often scarcely ∗Corresponding Author labeled. Furthermore, each biomedical domain can have distinct linguistic characteristics which are non-overlapping with those found in other biomed- ical domains (Lee et al., 2019). This linguistic challenge often diminishes the robustness of PLMs transferred from high-resource biomedical domains to low-resource ones (Giorgi and Bader, 2019). Given the premise, this paper focuses on adapt- ing PLMs for bioNER tasks learned from high- resource biomedical domains to low-resource ones. A potential solution is to inject the prior “experi- ence” to this adaptation process. Few works have explored this area such as Li et al. (2020a) and Li et al. (2020b), where the former followed the optimization/meta-learning strategy by (Finn et al., 2017) and the latter introduced a feature critic mod- ule similar to the work of Li et al. (2019). We show that simply incorporating the hard- ness information that each domain contributes to the learning of the bioNER LMs could signifi- cantly boost the adaptation performance of existing learning paradigms under various low-resource set- tings. This happens since the importance/hardness of biomedical domains can vary significantly, …

Content truncated to 500 words