Better Feature Integration for Named Entity Recognition

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3457–3469 June 6–11, 2021. ©2021 Association for Computational Linguistics 3457 Better Feature Integration for Named Entity Recognition Lu Xu1, 2, Zhanming Jie1, 3, Wei Lu1, Lidong Bing2 1 StatNLP Research Group, Singapore University of Technology and Design 2 DAMO Academy, Alibaba Group 3 ByteDance xu_lu@mymail.sutd.edu.sg, allan@bytedance.com luwei@sutd.edu.sg, l.bing@alibaba-inc.com Abstract It has been shown that named entity recogni- tion (NER) could benefit from incorporating the long-distance structured information cap- tured by dependency trees. We believe this is because both types of features – the contextual information captured by the linear sequences and the structured information captured by the dependency trees may complement each other. However, existing approaches largely focused on stacking the LSTM and graph neural net- works such as graph convolutional networks (GCNs) for building improved NER models, where the exact interaction mechanism be- tween the two different types of features is not very clear, and the performance gain does not appear to be significant. In this work, we pro- pose a simple and robust solution to incorpo- rate both types of features with our Synergized- LSTM (Syn-LSTM), which clearly captures how the two types of features interact. We con- duct extensive experiments on several standard datasets across four languages. The results demonstrate that the proposed model achieves better performance than previous approaches while requiring fewer parameters. Our fur- ther analysis demonstrates that our model can capture longer dependencies compared with strong baselines.1 1 Introduction Named entity recognition (NER) is one of the most fundamental and important tasks in natu- ral language processing (NLP). While the litera- ture (Peters et al., 2018; Akbik et al., 2018; De- vlin et al., 2019) largely focuses on training deep language models to improve the contextualized word representations, previous studies show that ∗ Lu Xu is under the Joint PhD Program between Alibaba and Singapore University of Technology and Design. The work was done when Zhanming Jie was a PhD student in Singapore University of Technology and Design. 1We make our code publicly available at https:// github.com/xuuuluuu/SynLSTM-for-NER. ORG ORG Precision Castparts Corp. , Portlan , will begin trading with the symbol PCP . PRODUCT ? Dependency Path: Corp. begin trading with symbol PCP Hybrid Paths: Corp. begin . PCP OR Corp. begin trading with the symbol PCP neighbor context Figure 1: A sentence annotated with dependency trees and named entities. The paths to connect two entities are shown below the sentence. the structured information such as interactions be- tween non-adjacent words can also be important for NER (Finkel et al., 2005; Jie et al., 2017; Aguilar and Solorio, 2019). However, sequence models such as bidirectional LSTM (Hochreiter and Schmidhuber, 1997) are not able to fully capture the long-range dependencies (Bengio, 2009). For instance, Figure 1 (top) shows one type of structured information in NER. The words “Precision Castparts Corp.” can be easily inferred as ORGANIZATION by its context (i.e., Corp.). However, the second entity …

Content truncated to 500 words