Learning with Noisy Labels for Sentence-level Sentiment Classification

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 6286–6292, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 6286 Learning with Noisy Labels for Sentence-level Sentiment Classification Hao Wang ♮, ‡ Bing Liu ‡, ∗ Chaozhuo Li § Yan Yang ♮ Tianrui Li ♮ ♮School of Information Science and Technology, Southwest Jiaotong University hwang@my.swjtu.edu.cn, {yyang; trli}@swjtu.edu.cn ‡Department of Computer Science, University of Illinois at Chicago liub@uic.edu §State Key Lab of Software Development Environment, Beihang University lichaozhuo@buaa.edu.cn Abstract Deep neural networks (DNNs) can fit (or even over-fit) the training data very well. If a DNN model is trained using data with noisy la- bels and tested on data with clean labels, the model may perform poorly. This paper stud- ies the problem of learning with noisy labels for sentence-level sentiment classification. We propose a novel DNN model called NETAB (as shorthand for convolutional neural NETworks with AB-networks) to handle noisy labels dur- ing training. NETAB consists of two convolu- tional neural networks, one with a noise tran- sition layer for dealing with the input noisy la- bels and the other for predicting ‘clean’ labels. We train the two networks using their respec- tive loss functions in a mutual reinforcement manner. Experimental results demonstrate the effectiveness of the proposed model. 1 Introduction It is well known that sentiment annotation or la- beling is subjective (Liu, 2012). Annotators often have many disagreements. This is especially so for crowd-workers who are not well trained. That is why one always feels that there are many errors in an annotated dataset. In this paper, we study whether it is possible to build accurate sentiment classifiers even with noisy-labeled training data. Sentiment classification aims to classify a piece of text according to the polarity of the sentiment ex- pressed in the text, e.g., positive or negative (Pang and Lee, 2008; Liu, 2012; Zhang et al., 2018). In this work, we focus on sentence-level sentiment classification (SSC) with labeling errors. As we will see in the experiment section, noisy labels in the training data can be highly damag- ing, especially for DNNs because they easily fit the training data and memorize their labels even when training data are corrupted with noisy labels ∗Corresponding author (Zhang et al., 2017). Collecting datasets annotated with clean labels is costly and time-consuming as DNN based models usually require a large num- ber of training examples. Researchers and practi- tioners typically have to resort to crowdsourcing. However, as mentioned above, the crowdsourced annotations can be quite noisy. Research on learning with noisy labels dates back to 1980s (Angluin and Laird, 1988). It is still vibrant today (Mnih and Hinton, 2012; Natarajan et al., 2013, 2018; Menon et al., 2015; Gao et al., 2016; Liu and Tao, 2016; Khetan et al., 2018; Zhan et al., 2019) as it is highly challenging. We will discuss the related work in the next section. This paper studies the problem of …

Content truncated to 500 words