DAGA: Data Augmentation with a Generation Approach for Low-resource Tagging Tasks

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 6045–6057, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 6045 DAGA: Data Augmentation with a Generation Approach for Low-resource Tagging Tasks Bosheng Ding∗12 Linlin Liu∗12 Lidong Bing2 Canasai Kruengkrai†3 Thien Hai Nguyen2 Shafiq Joty1 Luo Si2 Chunyan Miao1 1Nanyang Technological University, Singapore 2DAMO Academy, Alibaba Group 3National Institute of Informatics, Japan {bosheng.ding, linlin.liu, l.bing, thienhai.nguyen, luo.si}@alibaba-inc.com canasai@nii.ac.jp {srjoty, ascymiao}@ntu.edu.sg Abstract Data augmentation techniques have been widely used to improve machine learning per- formance as they enhance the generalization capability of models. In this work, to generate high quality synthetic data for low-resource tagging tasks, we propose a novel augmen- tation method with language models trained on the linearized labeled sentences. Our method is applicable to both supervised and semi-supervised settings. For the supervised settings, we conduct extensive experiments on named entity recognition (NER), part of speech (POS) tagging and end-to-end target based sentiment analysis (E2E-TBSA) tasks. For the semi-supervised settings, we evaluate our method on the NER task under the con- ditions of given unlabeled data only and unla- beled data plus a knowledge base. The results show that our method can consistently outper- form the baselines, particularly when the given gold training data are less.1 1 Introduction A large amount of training data is often essential for neural model performance, especially, for large net- works. Having more training data can help reduce overfitting and improve model robustness. How- ever, preparing a large amount of annotated data is usually costly, labor intensive and time-consuming. Data augmentation (Simard et al., 1998) is a useful technique for synthetic data generation, which is widely used in computer vision (Fawzi et al., 2016; ∗Equal contribution. Bosheng Ding and Linlin Liu are under the Joint PhD Program between Alibaba and Nanyang Technological University. †Work done while at DAMO Academy, Alibaba Group 1Our code is available at https://ntunlpsg. github.io/project/daga/ Figure 1: An example of labeled sentence linearization. All words and their tags are paired up by inserting tags before (or after) the words (O tags removed). D’Innocente et al., 2017; Wang et al., 2019) and speech (Schl¨uter and Grill, 2015; Ko et al., 2017). However, due to the complexity of language, it is more challenging to apply data augmentation techniques to natural language processing (NLP). Unlike computer vision and speech, where hand- crafted rules (such as rotation, cropping, masking, etc.) can be easily applied to transform original data, it is difficult to generalize such rules for lan- guages. Although simple distortion usually does not change the semantics of visual information, deleting or replacing a single word could com- pletely change the meaning of the sentence. One successful method for data augmentation in NLP is back translation (Sennrich et al., 2016; Fadaee et al., 2017; Dong et al., 2017; Yu et al., 2018), where a translation model is used to trans- late monolingual sentences from target language to source language to generate synthetic parallel sen- tences. Other successful methods include: system- atically reordering the …

Content truncated to 500 words