Learning from Miscellaneous Other-Class Words for Few-shot Named Entity Recognition

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 6236–6247 August 1–6, 2021. ©2021 Association for Computational Linguistics 6236 Learning from Miscellaneous Other-Class Words for Few-shot Named Entity Recognition Meihan Tong1, Shuai Wang2, Bin Xu1∗, Yixin Cao3, Minghui Liu1, Lei Hou1, Juanzi Li1 1Knowledge Engineering Laboratory, Tsinghua University, Beijing, China 2SLP Group, AI Technology Department, JOYY Inc, China 3S-Lab Nanyang Technological University, Singapore tongmeihan@gmail.com, wangshuai1@yy.com xubin@tsinghua.edu.cn, caoyixin2011@gmail.com liu-mh16@mails.tsinghua.edu.cn,greener2009@gmail.com lijuanzi@tsinghua.edu.cn Abstract Few-shot Named Entity Recognition (NER) exploits only a handful of annotations to iden- tify and classify named entity mentions. Pro- totypical network shows superior performance on few-shot NER. However, existing prototyp- ical methods fail to differentiate rich seman- tics in other-class words, which will aggravate overfitting under few shot scenario. To address the issue, we propose a novel model, Mining Undefined Classes from Other-class (MUCO), that can automatically induce different unde- fined classes from the other class to improve few-shot NER. With these extra-labeled unde- fined classes, our method will improve the dis- criminative ability of NER classifier and en- hance the understanding of predefined classes with stand-by semantic knowledge. Experi- mental results demonstrate that our model out- performs five state-of-the-art models in both 1- shot and 5-shots settings on four NER bench- marks. We will release the code upon accep- tance. The source code is released on https: //github.com/shuaiwa16/OtherClassNER.git. 1 Introduction Named Entity Recognition (NER) seeks to locate and classify named entities from sentences into predefined classes (Yadav and Bethard, 2019). Hu- mans can immediately recognize new entity types given just one or a few examples(Lake et al., 2015). Although neural NER networks have achieved su- perior performance when provided large-scale of training examples (Li et al., 2019), it remains a non-trivial task to learn from limited new samples, also known as few-shot NER (Fritzler et al., 2019). Traditional NER models, such as LSTM+CRF (Lample et al., 2016), fail in few-shot settings. They calculate the transition probability matrix based on statistics, which requires a large num- ber of data for optimization. Recently, prototypical ∗Corresponding author. Previous Methods Our Method Person Location O3 O2 O1 Predefined classes Undefined classes S1: Emeneya was born in local hospital and died in Paris O3 O1 O1 S3: The professor from the city studies mathematics O3 O2 O1 O1 S2: Newton is a polymath. He was born in Lincolnshire. O2 (a) (b) O O Person Location Figure 1: (a): Examples for undefined classes. (b): Dif- ferent ways to handle O class (single prototype vs. mul- tiple prototypes). network (Snell et al., 2017) shows potential on few- shot NER. The basic idea is to learn prototypes for each predefined entity class and an other class, then classify examples based on which prototypes they are closest to (Fritzler et al., 2019). Most existing studies focus on the predefined classes and lever- age the label semantic to reveal their dependency for enhancement (Hou et al., 2020). However, they ignore the massive semantics hidden …

Content truncated to 500 words