Instance-Based Learning of Span Representations: A Case Study through Named Entity Recognition

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6452–6459 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 6452 Instance-Based Learning of Span Representations: A Case Study through Named Entity Recognition Hiroki Ouchi1,2 Jun Suzuki2,1 Sosuke Kobayashi2,3 Sho Yokoi2,1 Tatsuki Kuribayashi2,4 Ryuto Konno2 Kentaro Inui2,1 1 RIKEN 2 Tohoku University 3 Preferred Networks, Inc. 4 Langsmith, Inc. hiroki.ouchi@riken.jp {jun.suzuki,sosk,yokoi,kuribayashi,ryuto,inui}@ecei.tohoku.ac.jp Abstract Interpretable rationales for model predictions play a critical role in practical applications. In this study, we develop models possessing inter- pretable inference process for structured pre- diction. Specifically, we present a method of instance-based learning that learns similarities between spans. At inference time, each span is assigned a class label based on its similar spans in the training set, where it is easy to understand how much each training instance contributes to the predictions. Through empir- ical analysis on named entity recognition, we demonstrate that our method enables to build models that have high interpretability without sacrificing performance. 1 Introduction Neural networks have contributed to performance improvements in structured prediction. Instead, the rationales underlying the model predictions are dif- ficult for humans to understand (Lei et al., 2016). In practical applications, interpretable rationales play a critical role for driving human’s decisions and promoting human-machine cooperation (Ribeiro et al., 2016). With this motivation, we aim to build models that have high interpretability without sac- rificing performance. As an approach to this chal- lenge, we focus on instance-based learning. Instance-based learning (Aha et al., 1991) is a machine learning method that learns similarities be- tween instances. At inference time, the class labels of the most similar training instances are assigned to the new instances. This transparent inference process provides an answer to the following ques- tion: Which points in the training set most closely resemble a test point or influenced the prediction? This is categorized into example-based explana- tions (Plumb et al., 2018; Baehrens et al., 2010). Recently, despite its preferable property, it has re- ceived little attention and been underexplored. This study presents and investigates an instance- based learning method for span representations. A span is a unit that consists of one or more linguis- tically linked words. Why do we focus on spans instead of tokens? One reason is relevant to perfor- mance. Recent neural networks can induce good span feature representations and achieve high per- formance in structured prediction tasks, such as named entity recognition (NER) (Sohrab and Miwa, 2018; Xia et al., 2019), constituency parsing (Stern et al., 2017; Kitaev et al., 2019), semantic role label- ing (SRL) (He et al., 2018; Ouchi et al., 2018) and coreference resolution (Lee et al., 2017). Another reason is relevant to interpretability. The tasks above require recognition of linguistic structure that consists of spans. Thus, directly classifying each span based on its representation is more inter- pretable than token-wise classification such as BIO tagging, which reconstructs each span label from the predicted token-wise BIO tags. Our method builds a feature space where spans with the …

Content truncated to 500 words