Data Augmentation for Cross-Domain Named Entity Recognition

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 5346–5356 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 5346 Data Augmentation for Cross-Domain Named Entity Recognition Shuguang Chen †, Gustavo Aguilar †, Leonardo Neves ∗ and Thamar Solorio † University of Houston † Snap Inc. ∗ {schen52, gaguilaralas, tsolorio}@uh.edu† lneves@snap.com∗ Abstract Current work in named entity recognition (NER) shows that data augmentation tech- niques can produce more robust models. How- ever, most existing techniques focus on aug- menting in-domain data in low-resource sce- narios where annotated data is quite limited. In contrast, we study cross-domain data aug- mentation for the NER task. We investigate the possibility of leveraging data from high- resource domains by projecting it into the low- resource domains. Specifically, we propose a novel neural architecture to transform the data representation from a high-resource to a low-resource domain by learning the patterns (e.g. style, noise, abbreviations, etc.) in the text that differentiate them and a shared fea- ture space where both domains are aligned. We experiment with diverse datasets and show that transforming the data to the low-resource domain representation achieves significant im- provements over only using data from high- resource domains. 1 1 Introduction Named entity recognition (NER) has seen signif- icant performance improvements with the recent advances of pre-trained language models (Akbik et al., 2019; Devlin et al., 2019). However, the high performance of such models usually relies on the size and quality of training data. When used un- der low-resource or even zero-resource scenarios, those models struggle to generalize over diverse domains (Fu et al., 2020), and the performance drops dramatically due to the lack of annotated data. Unfortunately, annotating more data is often expensive and time-consuming, and it requires ex- pert domain knowledge. Moreover, annotated data can quickly become outdated in domains where language changes rapidly (e.g, social media), lead- 1We release the code at https://github.com/ RiTUAL-UH/style_NER. Figure 1: Examples from Ontonotes 5.0 dataset and Temporal Twitter dataset. The language variations and abbreviations in the text from social media do- main make it clearly different from the formal text in newswire domain. ing to the temporal drift problem (Rijhwani and Preotiuc-Pietro, 2020). A common approach to alleviate the limitations mentioned above is data augmentation, where au- tomatically generated data can increase the size and diversity in the training set, while resulting in model performance gains. But data augmentation in the context of NER is still understudied. Ap- proaches that directly modify words in the training set (e.g, synonym replacement (Zhang et al., 2015) and word swap (Wei and Zou, 2019)) can inad- vertently result in incorrectly labeled entities after modification. Recent work in NER for low resource scenarios is promising (Dai and Adel, 2020; Ding et al., 2020) but it is limited to same domain set- tings and improvements decrease drastically with smaller sizes of training data. To facilitate research in this direction, we inves- tigate leveraging data from high-resource domains by projecting it into low-resource domains. Based on our …

Content truncated to 500 words