Improving Named Entity Recognition by External Context Retrieving and Cooperative Learning

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 1800–1812 August 1–6, 2021. ©2021 Association for Computational Linguistics 1800 Improving Named Entity Recognition by External Context Retrieving and Cooperative Learning Xinyu Wang⋄‡, Yong Jiang†∗, Nguyen Bach†, Tao Wang†, Zhongqiang Huang†, Fei Huang†, Kewei Tu⋄∗ ⋄School of Information Science and Technology, ShanghaiTech University Shanghai Engineering Research Center of Intelligent Vision and Imaging Shanghai Institute of Microsystem and Information Technology, Chinese Academy of Sciences University of Chinese Academy of Sciences †DAMO Academy, Alibaba Group {wangxy1,tukw}@shanghaitech.edu.cn, yongjiang.jy@alibaba-inc.com {nguyen.bach,leeo.wangt,z.huang,f.huang}@alibaba-inc.com Abstract Recent advances in Named Entity Recogni- tion (NER) show that document-level contexts can significantly improve model performance. In many application scenarios, however, such contexts are not available. In this paper, we propose to find external contexts of a sentence by retrieving and selecting a set of semanti- cally relevant texts through a search engine, with the original sentence as the query. We find empirically that the contextual represen- tations computed on the retrieval-based input view, constructed through the concatenation of a sentence and its external contexts, can achieve significantly improved performance compared to the original input view based only on the sentence. Furthermore, we can improve the model performance of both input views by Cooperative Learning, a training method that encourages the two input views to pro- duce similar contextual representations or out- put label distributions. Experiments show that our approach can achieve new state-of-the-art performance on 8 NER data sets across 5 do- mains.1 1 Introduction Pretrained contextual embeddings such as ELMo (Peters et al., 2018), Flair (Akbik et al., 2018) and BERT (Devlin et al., 2019) have significantly im- proved the accuracy of Named Entity Recognition (NER) models. Recent work (Devlin et al., 2019; Yu et al., 2020; Yamada et al., 2020) found that including document-level contexts of the target sen- tence in the input of contextual embeddings meth- ods can further boost the accuracy of NER models. However, there are a lot of application scenarios ∗Yong Jiang and Kewei Tu are the corresponding authors. ‡: This work was conducted when Xinyu Wang was interning at Alibaba DAMO Academy. 1Our code is publicly available at https://github. com/Alibaba-NLP/CLNER. senate democrats eliminated the nuclear option when they had the majority a few years ago , over republican objections . President Obama called for eliminating the legislative filibuster last month , which could occur if Democrats retake the Senate . Some Republicans say it s time to undo a wrong committed by Reid . Senate Republicans are considering using the nuclear option to end a potential Democratic filibuster and confirm Neil Gorsuch to the Supreme Court . Senate Republicans deployed the nuclear option on Wednesday to drastically reduce the time it takes to confirm hundreds of President Trump s nominees . Label: Group Label: Non Entity Input Sentence: Retrieved Texts: eliminated President option majority republican Some Senate . Some republican filibuster Democrats a Senate Republicans the Democratic Some Republicans Some Republicans …

Content truncated to 500 words