Prompt-Based Meta-Learning For Few-shot Text Classification

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 1342 - 1357 December 7-11, 2022 ©2022 Association for Computational Linguistics Prompt-Based Meta-Learning For Few-shot Text Classification Haoxing Zhang Xiaofeng Zhang Haibo Huang Lei Yu∗ Sino-French Engineer School, Beihang University, Beijing, China {mickael97, xiaofeng_z, huanghaibo, yulei}@buaa.edu.cn Abstract Few-shot Text Classification predicts the se- mantic label of a given text with a handful of supporting instances. Current meta-learning methods have achieved satisfying results in various few-shot situations. Still, they often require a large amount of data to construct many few-shot tasks for meta-training, which is not practical in real-world few-shot scenar- ios. Prompt-tuning has recently proved to be another effective few-shot learner by bridg- ing the gap between pre-train and downstream tasks. In this work, we closely combine the two promising few-shot learning methodolo- gies in structure and propose a Prompt-Based Meta-Learning (PBML) model to overcome the above meta-learning problem by adding the prompting mechanism. PBML assigns la- bel word learning to base-learners and tem- plate learning to meta-learner, respectively. Ex- perimental results show state-of-the-art perfor- mance on four text classification datasets un- der few-shot settings, with higher accuracy and good robustness. We demonstrate through low- resource experiments that our method alleviates the shortcoming that meta-learning requires too much data for meta-training. In the end, we use the visualization to interpret and verify that the meta-learning framework can help the prompt- ing method converge better. We release our code to reproduce our experiments1. 1 Introduction Humans can quickly learn new knowledge from a few examples, reflecting a high degree of intelli- gence. The meta-learning method was proposed to implement such human-like intelligence and achieved promising results in various few-shot sit- uations. It lifts the training unit from data point to task and trains a meta-learner over many tasks (called episodes) to grasp the few-shot learning ∗Corresponding author. 1Code at https://github.com/MGHZHANG/PBML The world's best fried foods. The topic is [MASK] Pretrained Language Model original text template Label Words politics food Class Set FOOD TECH technology sport POLITICS SPORTS Figure 1: An example of prompting. We concatenate the original text with a template. Then a pre-trained lan- guage model (PLM) will fill in [MASK] with words from a predefined label word set. A verbalizer will further map the predicted label word into the corresponding task label. (FSL) pattern. In this paper, we tackle the few- shot text classification, which aims to predict a given text’s label with only a handful of support instances. Current methods to solve the few-shot text classi- fication problem are based on meta-learning (Finn et al., 2017; Dong et al., 2020; Bao et al., 2020; Geng et al., 2020) or prompt-tuning (Brown et al., 2020; Gao et al., 2021; Zhao et al., 2021; Liu et al., 2021; Schick and Schütze, 2021). These two ap- proaches deal with "few-shot" from different per- spectives. On the one hand, the meta-learning al- gorithm aims to train a meta-learner across many episodes to extract transferable meta-knowledge. For each episode, the …

Content truncated to 500 words