Named Entity Recognition Only from Word Embeddings

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 8995–9005, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 8995 Named Entity Recognition Only from Word Embeddings Ying Luo, Hai Zhao ∗and Junlang Zhan Department of Computer Science and Engineering, Shanghai Jiao Tong University Key Laboratory of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering, Shanghai Jiao Tong University, Shanghai, China MoE Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University, Shanghai, China kingln@sjtu.edu.cn, zhaohai@cs.sjtu.edu.cn, longmr.zhan@sjtu.edu.cn Abstract Deep neural network models have helped named entity recognition achieve amazing per- formance without handcrafting features. How- ever, existing systems require large amounts of human annotated training data. Efforts have been made to replace human annotations with external knowledge (e.g., NE dictionary, part- of-speech tags), while it is another challenge to obtain such effective resources. In this work, we propose a fully unsupervised NE recognition model which only needs to take in- formative clues from pre-trained word embed- dings. We first apply Gaussian Hidden Markov Model and Deep Autoencoding Gaussian Mix- ture Model on word embeddings for entity span detection and type prediction, and then further design an instance selector based on reinforcement learning to distinguish positive sentences from noisy sentences and then refine these coarse-grained annotations through neu- ral networks. Extensive experiments on two CoNLL benchmark NER datasets (CoNLL- 2003 English dataset and CoNLL-2002 Span- ish dataset) demonstrate that our proposed light NE recognition model achieves remark- able performance without using any annotated lexicon or corpus. 1 Introduction Named Entity (NE) recognition is a major natural language processing task that intends to identify words or phrases that contain the names of PER (Person), ORG (Organization), LOC (Location), etc. Recent advances in deep neural models allow us to build reliable NE recognition systems (Lam- ple et al., 2016; Ma and Hovy, 2016; Liu et al., ∗ Corresponding author. This paper was partially sup- ported by National Key Research and Development Program of China (No. 2017YFB0304100), Key Projects of Na- tional Natural Science Foundation of China (U1836222 and 61733011), Huawei-SJTU long term AI project, Cutting-edge Machine reading comprehension and language model. 2018; Yang and Zhang, 2018; Luo et al., 2020; Luo and Zhao, 2020). However, these existing methods require large amounts of manually annotated data for training supervised models. There have been efforts to deal with the lack of annotation data in NE recognition, (Talukdar and Pereira, 2010) train a weak supervision model and use label propaga- tion methods to identify more entities of each type; (Shen et al., 2017) employ Deep Active Learning to efficiently select the set of samples for labeling, thus greatly reduce the annotation budget; (Ren et al., 2015; Shang et al., 2018; Fries et al., 2017; Yang et al., 2018b; Jie et al., 2019) use partially annotated data or external resources such as NE dictionary, knowledge base, POS tags as a replace- ment of hand-labeled data to train distant supervi- sion systems. However, these methods still have certain requirements for annotation resources. Un- …

Content truncated to 500 words