Feature-Dependent Confusion Matrices for Low-Resource NER Labeling with Noisy Labels

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 3554–3559, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 3554 Feature-Dependent Confusion Matrices for Low-Resource NER Labeling with Noisy Labels Lukas Lange1,2 Michael A. Hedderich2 1 Bosch Center for Artificial Intelligence, Renningen, Germany 2 Spoken Language Systems (LSV), Saarland University, Saarland Informatics Campus, Saarbr¨ucken, Germany {llange,mhedderich,dietrich.klakow}@lsv.uni-saarland.de Dietrich Klakow2 Abstract In low-resource settings, the performance of supervised labeling models can be improved with automatically annotated or distantly su- pervised data, which is cheap to create but of- ten noisy. Previous works have shown that significant improvements can be reached by injecting information about the confusion be- tween clean and noisy labels in this additional training data into the classifier training. How- ever, for noise estimation, these approaches either do not take the input features (in our case word embeddings) into account, or they need to learn the noise modeling from scratch which can be difficult in a low-resource set- ting. We propose to cluster the training data using the input features and then compute dif- ferent confusion matrices for each cluster. To the best of our knowledge, our approach is the first to leverage feature-dependent noise modeling with pre-initialized confusion matri- ces. We evaluate on low-resource named en- tity recognition settings in several languages, showing that our methods improve upon other confusion-matrix based methods by up to 9%. 1 Introduction Most languages, even with millions of speakers, have not been the center for natural language processing and are counted as low-resource for tasks like named entity recognition (NER). Sim- ilarly, even for high-resource languages, there ex- ists only few labeled data for most entity types be- yond person, location and organization. Distantly- or weakly-supervised approaches have been pro- posed to solve this issue, e.g., by using lists of entities for labeling raw text (Ratinov and Roth, 2009; Dembowski et al., 2017). This allows ob- taining large amounts of training data quickly and cheaply. Unfortunately, these labels often contain errors and learning with this noisily-labeled data is difficult and can even reduce overall performance (see, e.g. Fang and Cohn (2016)). A variety of ideas have been proposed to over- come the issues of noisy training data. One pop- ular approach is to estimate the relation between noisy and clean, gold-standard labels and use this noise model to improve the training procedure. However, most of these approaches only assume a dependency between the labels and do not take the features into account when modeling the la- bel noise. This may disregard important informa- tion. The global confusion matrix (Hedderich and Klakow, 2018) is a simple model which assumes that the errors in the noisy labels just depend on the clean labels. Our contributions are as follows: We propose to cluster the input words with the help of additional, unlabeled data. Based on this partition of the fea- ture space, we obtain different confusion matri- ces that describe …

Content truncated to 500 words