Sentence-Level Resampling for Named Entity Recognition

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2151 - 2165 July 10-15, 2022 ©2022 Association for Computational Linguistics Sentence-Level Resampling for Named Entity Recognition Xiaochen Wang University of North Carolina at Chapel Hill Chapel Hill, NC, USA xcwang@email.unc.edu Yue Wang University of North Carolina at Chapel Hill Chapel Hill, NC, USA wangyue@unc.edu Abstract As a fundamental task in natural language pro- cessing, named entity recognition (NER) aims to locate and classify named entities in unstruc- tured text. However, named entities are always the minority among all tokens in the text. This data imbalance problem presents a challenge to machine learning models as their learning ob- jective is usually dominated by the majority of non-entity tokens. To alleviate data imbalance, we propose a set of sentence-level resampling methods where the importance of each train- ing sentence is computed based on its tokens and entities. We study the generalizability of these resampling methods on a wide variety of NER models (CRF, Bi-LSTM, and BERT) across corpora from diverse domains (general, social, and medical texts). Extensive experi- ments show that the proposed methods improve performance of the evaluated NER models es- pecially on small corpora, frequently outper- forming sub-sentence-level resampling, data augmentation, and special loss functions such as focal and Dice loss.1 1 Introduction In natural language processing, named entity recog- nition (NER) is an important task both on its own and for numerous downstream tasks such as entity linking and question answering. NER has an in- herent data imbalance problem: named entities of interest are almost always the minority among ir- relevant (Other type) tokens in a text corpus. Table 1 shows the prevalent imbalanced nature of NER corpora from multiple domains. As shown in Table 1, entity tokens (tokens associated with any named entity) account for 3.9-16.6% of all tokens in any of these corpora. Within entity tokens, the most frequent entity type may cover 2-200 times more tokens than the least frequent entity type. At the sentence level, 23-85% sentences contain at least 1The source code is available at https://github. com/XiaoChen-W/NER_Adaptive_Resampling. one entity, suggesting that 15-77% sentences con- tain no entity at all. Data imbalance is even more severe in real-world bespoke NER tasks, which directly motivated this work. For example, given full-text articles from a medical subfield, domain experts may wish to recognize only those concepts related to specific as- pects of the subfield (e.g., symptoms and medicine related to a specific disease). Compared to all to- kens in the full text, extremely few tokens are anno- tated with any entity type. Because domain experts have limited availability, annotated corpus are usu- ally small in such tasks. As a result, some rare entity types may have less than 10 tokens across the corpus. Such severe data imbalance and scarcity makes many NER models suffer. Data imbalance in NER challenges machine learning-based models because their learning ob- jective is dominated by entities of the majority type (Other), …

Content truncated to 500 words