Simple Questions Generate Named Entity Recognition Datasets

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 6220 - 6236 December 7-11, 2022 ©2022 Association for Computational Linguistics Simple Questions Generate Named Entity Recognition Datasets Hyunjae Kim1 Jaehyo Yoo1 Seunghyun Yoon2 Jinhyuk Lee1∗ Jaewoo Kang1,3 1Korea University 2Adobe Research 3AIGEN Sciences {hyunjae-kim,jaehyoyoo,jinhyuk_lee,kangj}@korea.ac.kr syoon@adobe.com Abstract Recent named entity recognition (NER) models often rely on human-annotated datasets, requir- ing the significant engagement of professional knowledge on the target domain and entities. This research introduces an ask-to-generate approach that automatically generates NER datasets by asking questions in simple natural language to an open-domain question answer- ing system (e.g., “Which disease?”). Despite using fewer in-domain resources, our models, solely trained on the generated datasets, largely outperform strong low-resource models by an average F1 score of 19.4 for six popular NER benchmarks. Furthermore, our models provide competitive performance with rich-resource models that additionally leverage in-domain dictionaries provided by domain experts. In few-shot NER, we outperform the previous best model by an F1 score of 5.2 on three bench- marks and achieve new state-of-the-art perfor- mance. The code and datasets are available at https://github.com/dmis-lab/GeNER. 1 Introduction Named entity recognition (NER) is the task of ex- tracting named entities of specific types from text. An NER dataset essentially reflects the need to extract specific entity types. For instance, NCBI- disease (Do˘gan et al., 2014) was created for ex- tracting disease entities from text. Recent NER models have provided robust performance when trained on carefully designed human-annotated datasets (Lample et al., 2016; Li et al., 2020; Lee et al., 2020). However, suppose that we want to build an NER model for extracting bacteria or other specific types for which human-annotated datasets are insufficient. Whenever we extract such entity types, should we rely on professional knowledge to create new datasets? *JL currently works at Google Research. The collabora- tion started before he joined Google. Previous weakly supervised NER models (Shang et al., 2018b; Liang et al., 2020) tackled this prob- lem using rich in-domain dictionaries (e.g., The Comparative Toxicogenomics Database) and unla- beled training sentences (i.e., in-domain sentences), where entities in the dictionary are used to annotate the training sentences. However, these approaches easily fail in practice because in-domain dictionar- ies and sentences are often unavailable or expen- sive to construct for many entity types. It will be challenging to build NER models for enzyme or astronomical object entities without expert-level knowledge required for building dictionaries and searching for a large number of in-domain sen- tences for annotation. In this study, we introduce GeNER, an auto- mated dataset Generation framework for NER, which automatically constructs high-quality NER datasets. In particular, concrete needs for NER are described using simple natural language questions such as “Which [TYPE]?”, where [TYPE] is substi- tuted by the required entity type (e.g., “Which dis- ease?”). Such questions do not require professional knowledge of the target domain and allow even non-experts to easily build domain-specific NER datasets. Using a phrase retrieval model designed for open-domain question answering (QA) (Lee et …

Content truncated to 500 words