Named Entity Recognition through Deep Representation Learning and Weak Supervision

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 3828–3839 August 1–6, 2021. ©2021 Association for Computational Linguistics 3828 Named Entity Recognition through Deep Representation Learning and Weak Supervision Jerrod Parker The Vanguard Group University of Toronto jerrod parker@vanguard.com Shi Yu The Vanguard Group shi yu@vanguard.com Abstract Weakly supervised methods estimate the la- bels for a dataset using the predictions of sev- eral noisy supervision sources. Many machine learning practitioners have begun using weak supervision to more quickly and cheaply an- notate data compared to traditional manual la- beling. In this paper, we focus on the spe- cific problem of weakly supervised named en- tity recognition (NER) and propose an end- to-end model to learn optimal assignments of latent NER tags using observed tokens and weak labels provided by labeling functions. To capture the sequential dependencies between the latent and observed variables, we propose a sequential graphical model where the com- ponents are approximated using neural net- works. State-of-the-art contextual embeddings are used to further discriminate the quality of noisy weak labels in various contexts. Results of experiments on four public weakly super- vised named entity recognition datasets show a significant improvement in F1 score over re- cent approaches. 1 Introduction Many industries and organizations have collected large amounts of unlabeled text data that they want to make use of for various Natural Language Pro- cessing (NLP) applications. However, in many of these applications (named-entity recognition, ques- tion answering, text summarization, relation extrac- tion), obtaining a large number of labels can be prohibitively expensive, error-prone, or otherwise infeasible. Furthermore, domain adaptation (Han and Eisenstein, 2019), which is commonly used in scarce label settings, can often struggle in new emerging / specific domains that don’t have any closely related labeled datasets. Under the absence of a closely related labeled dataset, weakly supervised learning is often used as a cheaper, less time-consuming alternative to obtaining gold standard labels. The main idea of weak supervision is to approximate the true labels by integrating multiple sets of noisy training la- bels. Each set of noisy labels (commonly referred to as “weak labels”) is provided by a weak labeling function which often comes in the form of a knowl- edge base, heuristic, or pre-trained model. Weak supervision has obtained a lot of success in sev- eral NLP tasks containing approximately i.i.d. data such as topic classification (Bach et al., 2019), sen- timent analysis, and social media content tagging (Fu et al., 2020). Weak supervision on sequential data labeling problems such as NER is an emerging topic. Most current methods require either the time-consuming creation of additional heuristics such as ‘linking rules’(Safranchik et al., 2020) or ‘entity boundary detectors’(Fries et al., 2017), or assume that the ac- curacy of a weak labeler only depends on the true latent class (Safranchik et al., 2020; Lison et al., 2020; Fries et al., 2017). The latter is likely subop- timal since we would expect the accuracy to vary even within instances of the same class depending on the …

Content truncated to 500 words