Interpretability Analysis for Named Entity Recognition to Understand System Predictions and How They Can Improve

Interpretability Analysis for Named Entity Recognition to Understand System Predictions and How They Can Improve Oshin Agarwal University of Pennsylvania Department of Computer and Information Science oagarwal@seas.upenn.edu Yinfei Yang Google Research yinfeiy@google.com Byron C. Wallace Northeastern University Khoury College of Computer Sciences b.wallace@northeastern.edu Ani Nenkova University of Pennsylvania Department of Computer and Information Science nenkova@seas.upenn.edu Named entity recognition systems achieve remarkable performance on domains such as English news. It is natural to ask: What are these models actually learning to achieve this? Are they merely memorizing the names themselves? Or are they capable of interpreting the text and inferring the correct entity type from the linguistic context? We examine these questions by contrasting the performance of several variants of architectures for named entity recognition, with some provided only representations of the context as features. We experiment with GloVe- based BiLSTM-CRF as well as BERT. We find that context does influence predictions, but the main factor driving high performance is learning the named tokens themselves. Furthermore, we find that BERT is not always better at recognizing predictive contexts compared to a BiLSTM- CRF model. We enlist human annotators to evaluate the feasibility of inferring entity types from context alone and find that humans are also mostly unable to infer entity types for the majority of examples on which the context-only system made errors. However, there is room for improve- ment: A system should be able to recognize any named entity in a predictive context correctly Submission received: 28 April 2020; revised version received: 12 November 2020; accepted for publication: 8 December 2020. https://doi.org/10.1162/COLI a 00397 © 2021 Association for Computational Linguistics Published under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0) license Computational Linguistics Volume 47, Number 1 and our experiments indicate that current systems may be improved by such capability. Our human study also revealed that systems and humans do not always learn the same contextual clues, and context-only systems are sometimes correct even when humans fail to recognize the entity type from the context. Finally, we find that one issue contributing to model errors is the use of “entangled” representations that encode both contextual and local token information into a single vector, which can obscure clues. Our results suggest that designing models that explicitly operate over representations of local inputs and context, respectively, may in some cases improve performance. In light of these and related findings, we highlight directions for future work. 1. Introduction Named Entity Recognition (NER) is the task of identifying words and phrases in text that refer to a person, location, or organization name, or some finer subcategory of these types. NER systems work well on domains such as English news, achieving high performance on standard data sets like MUC-6 (Grishman and Sundheim 1996), CoNLL 2003 (Tjong Kim Sang and De Meulder 2003), and OntoNotes (Pradhan and Xue 2009). However, prior work has shown that the performance deteriorates on entities unseen in the training data (Augenstein, Derczynski, and Bontcheva 2017; Fu et al. 2020) and …

Content truncated to 500 words