Good Examples Make A Faster Learner: Simple Demonstration-based Learning for Low-resource NER

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 2687 - 2700 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Good Examples Make A Faster Learner Simple Demonstration-based Learning for Low-resource NER Dong-Ho Lee1, Akshen Kadakia1∗, Kangmin Tan1∗, Mahak Agarwal1, Xinyu Feng1, Takashi Shibuya2, Ryosuke Mitani2, Toshiyuki Sekiya2, Jay Pujara1, Xiang Ren1 1Department of Computer Science, University of Southern California 2R&D Center, Sony Group Corporation {dongho.lee,akshenhe,kangmint,mahakaga,xinyuf,jpujara,xiangren}@usc.edu {Takashi.Tak.Shibuya,Ryosuke.Mitani,Toshiyuki.Sekiya}@sony.com Abstract Recent advances in prompt-based learning have shown strong results on few-shot text classification by using cloze-style templates. Similar attempts have been made on named entity recognition (NER) which manually de- sign templates to predict entity types for ev- ery text span in a sentence. However, such methods may suffer from error propagation in- duced by entity span detection, high cost due to enumeration of all possible text spans, and omission of inter-dependencies among token labels in a sentence. Here we present a sim- ple demonstration-based learning method for NER, which lets the input be prefaced by task demonstrations for in-context learning. We perform a systematic study on demonstration strategy regarding what to include (entity ex- amples, with or without surrounding context), how to select the examples, and what tem- plates to use. Results on in-domain learning and domain adaptation show that the model’s performance in low-resource settings can be largely improved with a suitable demonstra- tion strategy (e.g., 4-17% improvement on 25 train instances). We also find that good demon- stration can save many labeled examples and consistency in demonstration contributes to better performance. 1 1 Introduction Neural sequence models have become the de facto approach for named entity recognition (NER) and have achieved state-of-the-art results on various NER benchmarks (Lample et al., 2016; Ma and Hovy, 2016; Liu et al., 2018). However, these data-hungry models often rely on large amounts of labeled data manually annotated by human experts, which are expensive and slow to collect (Huang et al., 2020; Ding et al., 2021b), especially for specialized domains (e.g., research papers). To ∗Authors contributed equally. 1https://github.com/INK-USC/fewNER improve NER performance on low-resource (label scarcity) settings, prior works seek auxiliary su- pervisions, such as entity dictionary (Peng et al., 2019; Shang et al., 2018; Yang et al., 2018; Liu et al., 2019) and labeling rules (Safranchik et al., 2020; Jiang et al., 2020), to either augment human- labeled data with pseudo-labeled data, or incor- porate meta information such as explanation (Lin et al., 2020; Lee et al., 2020, 2021), context (Wang et al., 2021), and prompts (Ding et al., 2021a; Cui et al., 2021) to facilitate training. However, such methods have the following challenges: (1) hu- man efforts to create auxiliary supervisions (e.g., dictionaries, rules, and explanations); (2) high com- putational cost to make predictions. For example, Ding et al. (2021a) shows effectiveness on entity type prediction given the entity span by construct- ing a prompt with the structure “[entity span] is [MASK]". However, when the entity span is not given, cloze-style prompts need to be …

Content truncated to 500 words