An Effective Label Noise Model for DNN Text Classification

Proceedings of NAACL-HLT 2019, pages 3246–3256 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 3246 An Effective Label Noise Model for DNN Text Classification Ishan Jindal1, Daniel Pressel2, Brian Lester2, and Matthew Nokleby1 1Wayne State University, Detroit MI 48202 2Interactions Digital Roots, Ann Arbor MI 48104 {ishan.jindal, matthew.nokleby}@wayne.edu, {dpressel, blester}@interactions.com Abstract Because large, human-annotated datasets suf- fer from labeling errors, it is crucial to be able to train deep neural networks in the pres- ence of label noise. While training image classification models with label noise have re- ceived much attention, training text classifica- tion models have not. In this paper, we pro- pose an approach to training deep networks that is robust to label noise. This approach in- troduces a non-linear processing layer (noise model) that models the statistics of the la- bel noise into a convolutional neural network (CNN) architecture. The noise model and the CNN weights are learned jointly from noisy training data, which prevents the model from overfitting to erroneous labels. Through ex- tensive experiments on several text classifica- tion datasets, we show that this approach en- ables the CNN to learn better sentence repre- sentations and is robust even to extreme label noise. We find that proper initialization and regularization of this noise model is critical. Further, by contrast to results focusing on large batch sizes for mitigating label noise for image classification, we find that altering the batch size does not have much effect on classifica- tion performance. 1 Introduction Deep Neural Networks (DNNs) have led to sig- nificant advances in the fields of computer vi- sion (He et al., 2016), speech processing (Graves et al., 2013) and natural language processing (Kim, 2014; Young et al., 2018; Devlin et al., 2018). To be effective, supervised DNNs rely on large amounts of carefully labeled training data. However, it is not always realistic to assume that example labels are clean. Humans make mistakes and, depending on the complexity of the task, there may be disagreement even among expert la- belers. Further, samples drawn from the class con- ditional densities with overlapping supports gives rise to the label noise in training datasets. To sup- port noisy labels in data, we need new training methods that can be used to train DNNs directly from the corrupted labels to significantly reduce human labeling efforts. Zhu and Wu (2004) per- form an extensive study on the effect of label noise on classification performance of a classifier and find that noise in input features is less important than noise in training labels. In this work, we add a noise model layer on top of our target model to account for label noise in the training set, following (Jindal et al., 2016; Sukhbaatar et al., 2014). We provide extensive experiments on several text classification datasets with artificially injected label noise. We study the effect of two different types of label noise; Uni- form label flipping (Uni), where a clean label is swapped with another label sampled uniformly …

Content truncated to 500 words