DiffusionNER: Boundary Diffusion for Named Entity Recognition

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 3875–3890 July 9-14, 2023 ©2023 Association for Computational Linguistics DiffusionNER: Boundary Diffusion for Named Entity Recognition Yongliang Shen1∗, Kaitao Song2†, Xu Tan2, Dongsheng Li2, Weiming Lu1†, Yueting Zhuang1 Zhejiang University1, Microsoft Research Asia2 {syl, luwm}@zju.edu.cn, {kaitaosong, xuta}@microsoft.com Abstract In this paper, we propose DIFFUSIONNER, which formulates the named entity recognition task as a boundary-denoising diffusion process and thus generates named entities from noisy spans. During training, DIFFUSIONNER grad- ually adds noises to the golden entity bound- aries by a fixed forward diffusion process and learns a reverse diffusion process to recover the entity boundaries. In inference, DIFFU- SIONNER first randomly samples some noisy spans from a standard Gaussian distribution and then generates the named entities by de- noising them with the learned reverse diffusion process. The proposed boundary-denoising dif- fusion process allows progressive refinement and dynamic sampling of entities, empower- ing DIFFUSIONNER with efficient and flexible entity generation capability. Experiments on multiple flat and nested NER datasets demon- strate that DIFFUSIONNER achieves compara- ble or even better performance than previous state-of-the-art models1. 1 Introduction Named Entity Recognition (NER) is a basic task of information extraction (Tjong Kim Sang and De Meulder, 2003), which aims to locate entity mentions and label specific entity types such as person, location, and organization. It is fundamen- tal to many structured information extraction tasks, such as relation extraction (Li and Ji, 2014; Miwa and Bansal, 2016) and event extraction (McClosky et al., 2011; Wadden et al., 2019). Most traditional methods (Chiu and Nichols, 2016) formulate the NER task into a sequence la- beling task by assigning a single label to each token. To accommodate the nested structure between en- tities, some methods (Ju et al., 2018; Wang et al., * This work was done when the first author was an intern at Microsoft Research Asia. † Corresponding author. 1 Our code will be available at https://github.com/ tricktreat/DiffusionNER. achieved his master's degree from the University of Sydney ~ Named Entities Noisy Spans Forward Process Reverse Process denoise ( 8, 9 | GPE ) ( 16, 18 | ORG ) ( 1, 9 | PER ) Premier of the western Canadian province of British Columbia { ( 4, 5 | LOC ) ( 3, 9 | GPE ) +² » N(0; 1) +² » N(0; 1) Figure 1: Boundary diffusion in named entity recogni- tion. The fixed forward diffusion process adds Gaussian noise to the entity boundaries at each timestep, and the noisy boundaries recover their original state by denois- ing with the learnable reverse diffusion process. For inference, the reverse diffusion process generates en- tity boundaries and performs entity typing based on the noisy spans sampled from the Gaussian distribution. 2020) further devise cascaded or stacked tagging strategies. Another class of methods treat NER as a classification task on text spans (Sohrab and Miwa, 2018; Eberts and Ulges, 2020), and assign labels to word pairs (Yu et al., 2020; …

Content truncated to 500 words