Divide and Denoise: Learning from Noisy Labels in Fine-Grained Entity Typing with Cluster-Wise Loss Correction

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 1997 - 2006 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Divide and Denoise: Learning from Noisy Labels in Fine-Grained Entity Typing with Cluster-Wise Loss Correction Kunyuan Pang1†, Haoyu Zhang2†, Jie Zhou3, Ting Wang1‡ 1School of Computer, National University of Defense Technology 2Artificial Intelligence Research Center, Defense Innovation Institute 3Shanghai Jiao Tong University {pangkunyuan10, zhanghaoyu10, tingwang}@nudt.edu.cn , sanny02@sjtu.edu.cn Abstract Fine-grained Entity Typing (FET) has made great progress based on distant supervision but still suffers from label noise. Existing FET noise learning methods rely on prediction dis- tributions in an instance-independent manner, which causes the problem of confirmation bias. In this work, we propose a clustering-based loss correction framework named Feature Cluster Loss Correction (FCLC), to address these two problems. FCLC first train a coarse backbone model as a feature extractor and noise estimator. Loss correction is then applied to each feature cluster, learning directly from the noisy labels. Experimental results on three public datasets show that FCLC achieves the best performance over existing competitive systems. Auxiliary experiments further demonstrate that FCLC is stable to hyperparameters and it does help miti- gate confirmation bias. We also find that in the extreme case of no clean data, the FCLC frame- work still achieves competitive performance. 1 Introduction Fine-grained entity typing (FET) is the task of clas- sifying named entity mentions in a sentence over the given class set (typically a hierarchical class structure as shown in Fig. 1. FET serves as an important component in many down-stream NLP applications, e.g., relation extraction (Liu et al., 2014), entity linking (Raiman and Raiman, 2018) and question answering (Dong et al., 2015). FET task has a more wide range of entity types (usu- ally over 100 classes) compared to entity typing, and hence neural-based FET systems require large- scale annotated training corpus. Recent studies apply distant supervision to label the corpora automatically by linking mentions to knowledge base entities and using all entity types †These authors contributed equally to this work and should be considered co-first authors. ‡Corresponding author: Ting Wang. Figure 1: An Example of noisy labels and feature space illustration in FET task. as the ground-truth labels. Although large-scale annotated data is provided, it brings about label noises in training. To overcome the problem of noisy label, some works directly pruned noisy in- stances (Gillick et al., 2014; Onoe and Durrett, 2019a). The others retain noisy training data but further improve by choosing (Ren et al., 2016a; Xu and Barbosa, 2018), weighting (Wu et al., 2019), and relabeling (Zhang et al., 2020) noisy labels using the prediction distribution. However, these noise combating methods have two major limitations. 1) They rely on the predic- tion distribution. As a result, they ought to cope with instance-agnostic noise better. The previous works expirically show (Zheng and Yang, 2021) that the prediction distribution is more likely to be affected by noisy instances and suffer from confir- mation bias. This bias …

Content truncated to 500 words