Automatic Creation of Named Entity Recognition Datasets by Querying Phrase Representations

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 7148–7163 July 9-14, 2023 ©2023 Association for Computational Linguistics Automatic Creation of Named Entity Recognition Datasets by Querying Phrase Representations Hyunjae Kim1 Jaehyo Yoo1 Seunghyun Yoon2 Jaewoo Kang1,3 1Korea University 2Adobe Research 3AIGEN Sciences {hyunjae-kim,jaehyoyoo,kangj}@korea.ac.kr syoon@adobe.com Abstract Most weakly supervised named entity recog- nition (NER) models rely on domain-specific dictionaries provided by experts. This approach is infeasible in many domains where dictionar- ies do not exist. While a phrase retrieval model was used to construct pseudo-dictionaries with entities retrieved from Wikipedia automatically in a recent study, these dictionaries often have limited coverage because the retriever is likely to retrieve popular entities rather than rare ones. In this study, we present a novel framework, HighGEN, that generates NER datasets with high-coverage pseudo-dictionaries. Specifi- cally, we create entity-rich dictionaries with a novel search method, called phrase embed- ding search, which encourages the retriever to search a space densely populated with various entities. In addition, we use a new verifica- tion process based on the embedding distance between candidate entity mentions and entity types to reduce the false-positive noise in weak labels generated by high-coverage dictionaries. We demonstrate that HighGEN outperforms the previous best model by an average F1 score of 4.7 across five NER benchmark datasets. 1 Introduction Named entity recognition (NER) models often re- quire a vast number of manual annotations for train- ing, which limits their utility in practice. In several studies, external resources such as domain-specific dictionaries have been employed as weak supervi- sion to reduce annotation costs (Shang et al., 2018; Liang et al., 2020; Meng et al., 2021). However, such dictionaries exist only for certain domains and building a dictionary for a new domain requires a high level of expertise and effort. To address this problem, a recent study proposed a framework called GeNER, which generates NER datasets without hand-crafted dictionaries (Kim et al., 2022). In GeNER, user questions that re- flect the needs for NER are received as inputs 22. 10. 11. 오전 10:54 motivating_band.html file:///C:/Users/vkfwl/Downloads/motivating_band.html 1/1 david cameron tony blair beijing vienna the beatles pink floyd 1 2 3 4 5 6 7 8 9 10 0 50 100 150 200 250 300 350 Politician City Band Figure 1: The most frequent ten entities in the top 1,000 phrases retrieved from the 2018-12-20 version of Wikipedia for the three questions: “Which politi- cian?”, “Which city?”, and “Which band?”. Due to a bias in the entity popularity (Chen et al., 2021), a current phrase retrieval model, DensePhrases (Lee et al., 2021), primarily returns popular entities, limiting the coverage of dictionaries. (e.g., “Which city?”), and an open-domain question- answering (QA) system, DensePhrases (Lee et al., 2021), is used to retrieve relevant phrases (i.e., answers) and evidence sentences from Wikipedia. The retrieved phrases constitute a ‘pseudo’ dictio- nary, which serves as weak supervision in place of hand-crafted dictionaries. The evidence sen- tences are annotated based on string matching with the pseudo …

Content truncated to 500 words