Local Additivity Based Data Augmentation for Semi-supervised NER

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 1241–1251, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 1241 Local Additivity Based Data Augmentation for Semi-supervised NER Jiaao Chen∗, Zhenghui Wang∗, Ran Tian1, Zichao Yang2, Diyi Yang Georgia Institute of Technology, 1ASIT Japan / Google, 2Citadel Securities {jchen896,zhwang,dyang888}@gatech.edu Abstract Named Entity Recognition (NER) is one of the first stages in deep language understanding yet current NER models heavily rely on human- annotated data. In this work, to alleviate the de- pendence on labeled data, we propose a Local Additivity based Data Augmentation (LADA) method for semi-supervised NER, in which we create virtual samples by interpolating se- quences close to each other. Our approach has two variations: Intra-LADA and Inter- LADA, where Intra-LADA performs interpo- lations among tokens within one sentence, and Inter-LADA samples different sentences to in- terpolate. Through linear additions between sampled training data, LADA creates an in- finite amount of labeled data and improves both entity and context learning. We further extend LADA to the semi-supervised setting by designing a novel consistency loss for un- labeled data. Experiments conducted on two NER benchmarks demonstrate the effective- ness of our methods over several strong base- lines. We have publicly released our code at https://github.com/GT-SALT/LADA. 1 Introduction Named Entity Recognition (NER) that aims to de- tect the semantic category of entities (e.g., per- sons, locations, organizations) in unstructured text (Nadeau and Sekine, 2007), is an essential pre- requisite for many NLP applications. Being one of the most fundamental and classic sequence la- beling tasks in NLP, there have been extensive re- search from traditional statistical models like Hid- den Markov Models (Zhou and Su, 2002) and Con- ditional Random Fields (Lafferty et al., 2001a), to neural network based models such as LSTM- CRF (Lample et al., 2016a) and BLSTM-CNN- CRF (Ma and Hovy, 2016), and to recent pre- ∗Equal contribution. training and fine-tuning methods like ELMO (Pe- ters et al., 2018a), Flair (Akbik et al., 2018) and BERT (Devlin et al., 2019). However, most of those models still heavily rely on abundant anno- tated data to yield the state-of-the-art results (Lin et al., 2020), making them hard to be applied into new domains (e.g., social media, medical context or low-resourced languages) that lack labeled data. Different kinds of data augmentation approaches have been designed to alleviate the dependency on labeled data for many NLP tasks, and can be categorized into two broad classes: (1) adversar- ial attacks at token-levels such as word substitu- tions (Kobayashi, 2018; Wei and Zou, 2019) or adding noise (Lakshmi Narayan et al., 2019), (2) paraphrasing at sentence-levels such as back trans- lations (Xie et al., 2019) or submodular optimized models (Kumar et al., 2019). The former has al- ready been used for NER but struggles to create diverse augmented samples with very few word re- placements. Despite being widely utilized in many NLP tasks like text classification, the latter often fails to maintain the labels at the token-level in those paraphrased …

Content truncated to 500 words