Hero-Gang Neural Model For Named Entity Recognition

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1924 - 1936 July 10-15, 2022 ©2022 Association for Computational Linguistics Hero-Gang Neural Model For Named Entity Recognition Jinpeng Hu♡, Yaling Shen♡, Yang Liu♡ Xiang Wan♡♢†, Tsung-Hui Chang♡† ♡Shenzhen Research Institute of Big Data, The Chinese University of Hong Kong, Shenzhen, Guangdong, China ♢Pazhou Lab, Guangzhou, 510330, China {jinpenghu, yalingshen, yangliu5}@link.cuhk.edu.cn wanxiang@sribd.cn changtsunghui@cuhk.edu.cn Abstract Named entity recognition (NER) is a funda- mental and important task in NLP, aiming at identifying named entities (NEs) from free text. Recently, since the multi-head attention mecha- nism applied in the Transformer model can ef- fectively capture longer contextual information, Transformer-based models have become the mainstream methods and have achieved signif- icant performance in this task. Unfortunately, although these models can capture effective global context information, they are still limited in the local feature and position information ex- traction, which is critical in NER. In this paper, to address this limitation, we propose a novel Hero-Gang Neural structure (HGN), including the Hero and Gang module, to leverage both global and local information to promote NER. Specifically, the Hero module is composed of a Transformer-based encoder to maintain the advantage of the self-attention mechanism, and the Gang module utilizes a multi-window re- current module to extract local features and position information under the guidance of the Hero module. Afterward, the proposed multi- window attention effectively combines global information and multiple local features for pre- dicting entity labels. Experimental results on several benchmark datasets demonstrate the ef- fectiveness of our proposed model.1 1 Introduction Named entity recognition (NER) is one of the most important and fundamental research topics in natu- ral language processing (NLP), which recognizes named entities (NEs), such as person, location, disease from raw text. NER has attracted sub- stantial attention in the past decades owing to its importance in downstream tasks, e.g., knowledge †Corresponding author. 1Our code is released at https://github.com/ jinpeng01/HGN. graph construction (Bosselut et al., 2019), question- answering (Pergola et al., 2021), and relation ex- traction (He et al., 2019). In the early stage, the popular methods for solv- ing NER are some traditional machine learning methods, e.g., Hidden Markov Model (HMM) (Morwal et al., 2012) and conditional random field (CRF) (Mozharova and Loukachevitch, 2016), which require extracting features manually, mak- ing the process inefficient and time-consuming. With the breakthrough of recurrent neural networks (RNN) in NLP, Long short-term memory (LSTM) (Hochreiter et al., 1997) and Gated Recurrent Unit (GRU) (Cho et al., 2014) have become mainstream methods for this task and have achieved promis- ing results since neural networks can automatically extract features from the sequence and also take each token’s position information into considera- tion (Lample et al., 2016; Chiu and Nichols, 2016; Huang et al., 2015). Nevertheless, RNN fails to perform well with long sequences due to the gra- dients exploding and vanishing. In recent years, Transformer-based models (Vaswani et al., 2017) have become mainstream methods because these models …

Content truncated to 500 words