Learning to Denoise Distantly-Labeled Data for Entity Typing

Proceedings of NAACL-HLT 2019, pages 2407–2417 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 2407 Learning to Denoise Distantly-Labeled Data for Entity Typing Yasumasa Onoe and Greg Durrett Department of Computer Science The University of Texas at Austin {yasumasa, gdurrett}@cs.utexas.edu Abstract Distantly-labeled data can be used to scale up training of statistical models, but it is typi- cally noisy and that noise can vary with the distant labeling technique. In this work, we propose a two-stage procedure for handling this type of data: denoise it with a learned model, then train our final model on clean and denoised distant data with standard super- vised training. Our denoising approach con- sists of two parts. First, a filtering function discards examples from the distantly labeled data that are wholly unusable. Second, a rela- beling function repairs noisy labels for the re- tained examples. Each of these components is a model trained on synthetically-noised exam- ples generated from a small manually-labeled set. We investigate this approach on the ultra- fine entity typing task of Choi et al. (2018). Our baseline model is an extension of their model with pre-trained ELMo representations, which already achieves state-of-the-art perfor- mance. Adding distant data that has been de- noised with our learned models gives further performance gains over this base model, out- performing models trained on raw distant data or heuristically-denoised distant data. 1 Introduction With the rise of data-hungry neural network mod- els, system designers have turned increasingly to unlabeled and weakly-labeled data in order to scale up model training. For information extrac- tion tasks such as relation extraction and entity typing, distant supervision (Mintz et al., 2009) is a powerful approach for adding more data, using a knowledge base (Del Corro et al., 2015; Ra- binovich and Klein, 2017) or heuristics (Ratner et al., 2016; Hancock et al., 2018) to automatically label instances. One can treat this data just like any other supervised data, but it is noisy; more effec- tive approaches employ specialized probabilistic models (Riedel et al., 2010; Ratner et al., 2018a), capturing its interaction with other supervision (Wang and Poon, 2018) or breaking down aspects of a task on which it is reliable (Ratner et al., 2018b). However, these approaches often require sophisticated probabilistic inference for training of the final model. Ideally, we want a technique that handles distant data just like supervised data, so we can treat our final model and its training proce- dure as black boxes. This paper tackles the problem of exploiting weakly-labeled data in a structured setting with a two-stage denoising approach. We can view a dis- tant instance’s label as a noisy version of a true un- derlying label. We therefore learn a model to turn a noisy label into a more accurate label, then ap- ply it to each distant example and add the resulting denoised examples to the supervised training set. Critically, the denoising model can condition on both the example and its noisy label, allowing it to fully …

Content truncated to 500 words