Self-Adaptive Named Entity Recognition by Retrieving Unstructured Knowledge

Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 3193–3205 May 2-6, 2023 ©2023 Association for Computational Linguistics Self-Adaptive Named Entity Recognition by Retrieving Unstructured Knowledge Kosuke Nishida†‡ Naoki Yoshinaga§ Kyosuke Nishida† †NTT Human Informatics Laboratories, NTT Corporation ‡The University of Tokyo §Institute of Industrial Science, the University of Tokyo †{kosuke.nishida.ap, kyosuke.nishida.rx}@hco.ntt.co.jp §ynaga@iis.u-tokyo.ac.jp Abstract Although named entity recognition (NER) helps us to extract domain-specific entities from text (e.g., artists in the music domain), it is costly to create a large amount of training data or a struc- tured knowledge base to perform accurate NER in the target domain. Here, we propose self- adaptive NER, which retrieves external knowl- edge from unstructured text to learn the usages of entities that have not been learned well. To retrieve useful knowledge for NER, we design an effective two-stage model that retrieves un- structured knowledge using uncertain entities as queries. Our model predicts the entities in the input and then finds those of which the prediction is not confident. Then, it retrieves knowledge by using these uncertain entities as queries and concatenates the retrieved text to the original input to revise the prediction. Ex- periments on CrossNER datasets demonstrated that our model outperforms strong baselines by 2.35 points in F1 metric. 1 Introduction Named entity recognition (NER) helps us to ex- tract entities from text in various domains such as biomedicine (Kim et al., 2003), disease (Do˘gan et al., 2014), and COVID-19 (Wang et al., 2020). However, accurate neural NER requires a massive amount of training data (Chiu and Nichols, 2016; Ma and Hovy, 2016; Yadav and Bethard, 2018). As well, the annotation of a domain-specific NER dataset costs a lot of money because it requires the involvement of domain experts. To compensate for the lack of training data in NER, researchers have utilized external knowledge. Traditional feature-based NER uses features based on gazetteers or name lists (Florian et al., 2003; Cohen and Sarawagi, 2004; Luo et al., 2015) as external knowledge. Although recent neural NER methods can even benefit from gazetteers and name lists (Seyler et al., 2018; Liu et al., 2019; Mengge Model Unstructured KB Input Entity Prediction Entity Candidates Entity-Level Query Textual Knowledge Self- Adaptation Figure 1: Concept of self-adaptive NER: the model pre- dicts entity candidates to conduct entity-level retrieval from the unstructured KB; then it revises the prediction with reference to the retrieved knowledge. et al., 2020), only a few domains with struc- tured knowledge bases (gazetteers) have this merit. Thus, several studies have resorted to using raw text (unstructured knowledge) to perform weakly- supervised learning on general-domain structured knowledge (Cao et al., 2019; Mengge et al., 2020; Liu et al., 2021a). In this paper, we explore the potential of uti- lizing unstructured knowledge in the NER task by referring to it at inference time. Our basic idea is inspired by recent retrieval-augmented language models (LMs) (Guu et al., 2020). These models are pre-trained with retrieval-augmented masked language model (MLM), so that …

Content truncated to 500 words