Prompt-based Text Entailment for Low-Resource Named Entity Recognition

Proceedings of the 29th International Conference on Computational Linguistics, pages 1896–1903 October 12–17, 2022. 1896 Prompt-based Text Entailment for Low-Resource Named Entity Recognition Dongfang Li1, Baotian Hu1∗, Qingcai Chen1,2∗ 1Harbin Institute of Technology (Shenzhen), Shenzhen, China 2Peng Cheng Laboratory, Shenzhen, China crazyofapple@gmail.com, {hubaotian, qingcai.chen}@hit.edu.cn Abstract Pre-trained Language Models (PLMs) have been applied in NLP tasks and achieve promis- ing results. Nevertheless, the fine-tuning pro- cedure needs labeled data of the target domain, making it difficult to learn in low-resource and non-trivial labeled scenarios. To address these challenges, we propose Prompt-based Text En- tailment (PTE) for low-resource named entity recognition, which better leverages knowledge in the PLMs. We first reformulate named entity recognition as the text entailment task. The original sentence with entity type-specific prompts is fed into PLMs to get entailment scores for each candidate. The entity type with the top score is then selected as final label. Then, we inject tagging labels into prompts and treat words as basic units instead of n- gram spans to reduce time complexity in gener- ating candidates by n-grams enumeration. Ex- perimental results demonstrate that the pro- posed method PTE achieves competitive per- formance on the CoNLL03 dataset, and bet- ter than fine-tuned counterparts on the MIT Movie and Few-NERD dataset in low-resource settings. 1 Introduction Recently, Pre-trained Language Models (PLMs) have achieved promising improvement on several NLP tasks (Devlin et al., 2019; Liu et al., 2019; Lan et al., 2020). Nevertheless, fine-tuning language models still needs a moderate number of labeled data for downstream tasks. When difficulties re- sult in limited labeled data available, the trained model shows large variance in downstream per- formance under full fine-tuning (Mosbach et al., 2021; Le Scao and Rush, 2021). For example, labeling technical and professional terms can be time-consuming and labor-intensive in medical sce- narios. Moreover, crowd-sourced annotation is also limited by the reality of existing samples (e.g., ∗ Corresponding authors when online health assistants are applied to rare diseases). To address learning challenges in these low- resource scenarios, researchers find that PLMs can learn well by prompt-based learning (Schick and Schütze, 2021a,b; Tam et al., 2021). Prompt-based learning models the probability of text directly; it does not need an extra fully-connected layer usually used by fine-tuning. The main idea is to reformu- late NLP tasks as cloze-style question answering for better using the knowledge in PLMs. The model predicts the word probability of masked positions and then derives the final output via mapping re- lations between words and labels. Previous works have shown the ability of prompt-based learning under low-resource settings (Schick and Schütze, 2021a,b; Schick et al., 2020; Lester et al., 2021). For example, some prompt-based works have ex- plored in classification and generation tasks where it is relatively easy to reformulate into cloze-style tasks (cf. Section 4). Nevertheless, the application to Named Entity Recognition (NER) still poses challenges for current methods. Unlike text clas- sification and text generation, NER is the task of identifying named entities (e.g., person name, loca- tion) in …

Content truncated to 500 words