Counterfactual Generator: A Weakly-Supervised Method for Named Entity Recognition

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 7270–7280, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 7270 Counterfactual Generator: A Weakly-Supervised Method for Named Entity Recognition Xiangji Zeng, Yunliang Li, Yuchen Zhai, Yin Zhang∗ Zhejiang University, Hangzhou, China {zengxiangji,liyunliang,zhaiyuchen,zhangyin98}@zju.edu.cn Abstract Past progress on neural models has proven that named entity recognition is no longer a prob- lem if we have enough labeled data. How- ever, collecting enough data and annotating them are labor-intensive, time-consuming, and expensive. In this paper, we decompose the sentence into two parts: entity and context, and rethink the relationship between them and model performance from a causal perspective. Based on this, we propose the Counterfactual Generator, which generates counterfactual ex- amples by the interventions on the existing observational examples to enhance the origi- nal dataset. Experiments across three datasets show that our method improves the generaliza- tion ability of models under limited observa- tional examples. Besides, we provide a theo- retical foundation by using a structural causal model to explore the spurious correlations be- tween input features and output labels. We in- vestigate the causal effects of entity or context on model performance under both conditions: the non-augmented and the augmented. Inter- estingly, we find that the non-spurious corre- lations are more located in entity representa- tion rather than context representation. As a result, our method eliminates part of the spu- rious correlations between context representa- tion and output labels. The code is available at https://github.com/xijiz/cfgen. 1 Introduction The natural language processing community has witnessed the paradigm shift from small data to big data, such as transformer (Vaswani et al., 2017) and its successors. It is not surprising that machine learning methods can easily surpass human perfor- mance if sufficient data is available (Wang et al., 2018). However, data acquisition is a challeng- ing task for some special domains. For example, ∗*Corresponding Author Context 1 Context 1 Context 1 Context 1 Context 2 Context 2 Context 2 Context 2 Entity 1 Entity 1 Entity 2 Entity 2 Observational example 1 Pre-intervention Post-intervention Observational example 2 Counterfactual example 2 Counterfactual example 1 Figure 1: Interventions on observational examples of named entity recognition. More details can be found in Figure 3 medical concept normalization, a basic subtask of named entity recognition (NER) in the medical area, has always been troubled by lack of enough Electronic Health Records due to the privacy pro- tection. Small data with selection biases (Torralba and Efros, 2011) often induce the poor performance of machine learning models on inputs whose dis- tribution is different from that of training data, which yet seems trivial to humans. The same issues are also mentioned in terms like dataset bias, model robustness, and real understanding. In natural language inference, models trained on hypotheses-only (vs hypotheses-premises) can out- perform a majority-class baseline (Poliak et al., 2018; Gururangan et al., 2018). In reading com- prehension, models trained on question-only or passage-only (vs question-passage) still achieve high accuracy (Kaushik and Lipton, 2018), models predicted on …

Content truncated to 500 words