Inverse is Better! Fast and Accurate Prompt for Few-shot Slot Tagging

Findings of the Association for Computational Linguistics: ACL 2022, pages 637 - 647 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Inverse is Better! Fast and Accurate Prompt for Few-shot Slot Tagging Yutai Hou∗, Cheng Chen∗, Xianzhen Luo, Bohan Li, Wanxiang Che† Research Center for Social Computing and Information Retrieval, Harbin Institute of Technology {ythou, cchen, xzluo, bhli, car}@ir.hit.edu.cn Abstract Prompting methods recently achieve impres- sive success in few-shot learning. These meth- ods modify input samples with prompt sen- tence pieces, and decode label tokens to map samples to corresponding labels. However, such a paradigm is very inefficient for the task of slot tagging. Since slot tagging samples are multiple consecutive words in a sentence, the prompting methods have to enumerate all n-grams token spans to find all the possible slots, which greatly slows down the predic- tion. To tackle this, we introduce an inverse paradigm for prompting. Different from the classic prompts mapping tokens to labels, we reversely predict slot values given slot types. Such inverse prompting only requires a one- turn prediction for each slot type and greatly speeds up the prediction. Besides, we propose a novel Iterative Prediction Strategy, from which the model learns to refine predictions by consid- ering the relations between different slot types. We find, somewhat surprisingly, the proposed method not only predicts faster but also signif- icantly improves the effect (improve over 6.1 F1-scores on 10-shot setting) and achieves new state-of-the-art performance. 1 Introduction Few-shot learning (FSL) aims at learning a model from only a few examples and is regarded as one of the key steps toward more human-like artificial intelligence (Wang et al., 2020). Recently, prompt- based methods achieve impressive results and show promising prospects for few-shot learning of Natu- ral Language Processing (NLP) (Liu et al., 2021a; Zhao et al., 2021). Prompt-based methods reformulate a target task into the language modeling problem, which takes advantages of the powerful pretrained Language Models (LM) (Devlin et al., 2019; Liu et al., 2019; *Equal contributions. †Corresponding author. TPG 2 none beijing none tomorrow morning 2TPG new yo none departure refers to “ Original Input ” refers to “ Original Input ” arrival time refers to “ Original Input ” refers to “ Original Input ” price tomorrow morning . “ Original Input ” time refers to . departure refers to beijing . refers to price Mapped Labels Original Labels To.Loc Time Price From.Loc time price departure arrival Original Input: book a flight from beijing to new york tomorrow morning arrival . “ Original Input ” refers to tomorrow morning . departure refers to beijing . refers to time 1. 2. 主图 … Query LM × 55 [CLS] is a [MASK] entity . [SEP] Original Input book [CLS] is a [MASK] entity . [SEP] Original Input book a [CLS] is a [MASK] entity . [SEP] Original Input morning Original Input: book a flight from beijing to new york tomorrow morning time price departure arrival none TPG 2 Query LM × 4 refers to “ Original …

Content truncated to 500 words