Robust Self-Augmentation for Named Entity Recognition with Meta Reweighting

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4049 - 4060 July 10-15, 2022 ©2022 Association for Computational Linguistics Robust Self-Augmentation for Named Entity Recognition with Meta Reweighting Linzhi Wu1∗, Pengjun Xie∗, Jie Zhou2∗, Meishan Zhang3† , Chunping Ma, Guangwei Xu, Min Zhang3 1School of New Media and Communication, Tianjin University 2School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University 3Institute of Computing and Intelligence, Harbin Institute of Technology (Shenzhen) {tjuwlz2020, machunpingjj}@163.com, sanny02@sjtu.edu.cn {xpjandy, mason.zms, ahxgwOnePiece}@gmail.com, zhangmin2021@hit.edu.cn Abstract Self-augmentation has received increasing re- search interest recently to improve named en- tity recognition (NER) performance in low- resource scenarios. Token substitution and mixup are two feasible heterogeneous self- augmentation techniques for NER that can achieve effective performance with certain spe- cialized efforts. Noticeably, self-augmentation may introduce potentially noisy augmented data. Prior research has mainly resorted to heuristic rule-based constraints to reduce the noise for specific self-augmentation methods individually. In this paper, we revisit these two typical self-augmentation methods for NER, and propose a unified meta-reweighting strat- egy for them to achieve a natural integration. Our method is easily extensible, imposing little effort on a specific self-augmentation method. Experiments on different Chinese and English NER benchmarks show that our token substi- tution and mixup method, as well as their in- tegration, can achieve effective performance improvement. Based on the meta-reweighting mechanism, we can enhance the advantages of the self-augmentation techniques without much extra effort. 1 Introduction Named entity recognition (NER), which aims to extract predefined named entities from a piece of unstructured text, is a fundamental task in the nat- ural language processing (NLP) community, and has been studied extensively for several decades (Hammerton, 2003; Huang et al., 2015; Chiu and Nichols, 2016; Ma and Hovy, 2016). Recently, supervised sequence labeling neural models have been exploited most popularly for NER, leading to state-of-the-art (SOTA) performance (Zhang and Yang, 2018; Li et al., 2020a; Ma et al., 2020). Although great progress has been made, devel- oping an effective NER model usually requires a ∗Equal contributions. †Corresponding author. Feature Representation CRF Encoder Decoder Output Token Substitution Mixup Self-Augmentation Meta Reweighting Input Figure 1: The main idea of our work, where the two heterogeneous self-augmentation methods (i.e., token substitution and mixup) are integrated by a unified meta reweighting framework. large-scale and high-quality labeled training corpus, which is often difficult to be obtained in real-world scenarios due to the expensive and time-consuming annotations by human experts. Moreover, it would be extremely serious because the target language, target domain, and the desired entity type could all be infinitely varied. As a result, the low-resource setting with only a small amount of annotated cor- pus available is far more common in practice, even though it may result in significant performance degradation due to the overfitting problem. Self-augmentation is a prospective solution to this problem, which has received widespread at- tention (Zhang et al., 2018; Wei and Zou, 2019; Dai and Adel, 2020; Chen et …

Content truncated to 500 words