Automatic Multi-Label Prompting: Simple and Interpretable Few-Shot Classification

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5483 - 5492 July 10-15, 2022 ©2022 Association for Computational Linguistics Automatic Multi-Label Prompting: Simple and Interpretable Few-Shot Classification Han Wang1∗, Canwen Xu2∗†, Julian McAuley2 1New York University, 2University of California, San Diego 1hwang@nyu.edu, 2{cxu,jmcauley}@ucsd.edu Abstract Prompt-based learning (i.e., prompting) is an emerging paradigm for exploiting knowledge learned by a pretrained language model. In this paper, we propose Automatic Multi-Label Prompting (AMuLaP), a simple yet effective method to automatically select label mappings for few-shot text classification with prompt- ing. Our method exploits one-to-many label mappings and a statistics-based algorithm to select label mappings given a prompt tem- plate. Our experiments demonstrate that AMu- LaP achieves competitive performance on the GLUE benchmark without human effort or ex- ternal resources.1 1 Introduction Since the release of GPT-3 (Brown et al., 2020), several studies have focused on exploiting pre- trained language models with only a few training examples (Brown et al., 2020; Gao et al., 2021; Shin et al., 2020). These works demonstrate the potential of using natural language prompts to en- courage the model to recall similar patterns in its training corpus and thus make accurate predic- tions. This setting of few-shot learning is closer to how humans learn to solve a task, often without many examples as in a traditional deep learning paradigm. The use of prompts can strengthen the explicit connection between input and output, help- ing the model exploit the knowledge learned from pretraining in a better way. Furthermore, recent works (Schick and Schütze, 2021a,b; Gao et al., 2021) show that prompts can also help the model generalize better in fine-tuning. Prompt-based learning (i.e., prompting) aims to use a template to convert the original input into a prompt-based input with some unfilled masked ∗Equal contribution. †To whom correspondence should be addressed. 1The code is available at https://github.com/ HanNight/AMuLaP. tokens, and then use the pretrained language model to fill these masked tokens, and finally the tokens filled into these slots are mapped to the correspond- ing labels as the final output. In prompting, the design of prompts often plays an important role. Many attempts have been made in this emerging direction of prompt engineering (Shin et al., 2020; Gao et al., 2021). Meanwhile, finding a good map- ping from the original task labels to tokens (i.e., label engineering) is also critical to few-shot per- formance, as found in Schick et al. (2020); Gao et al. (2021). However, manually assigning the label mapping requires human expertise with trial and error. One may argue that the same effort can be used to label more supervised data for a con- ventional deep learning pipeline. Thus, an efficient automatic label mapping method is desirable. In this paper, we aim to design a method that can automatically find a good label mapping to save human effort from label engineering. We propose Automatic Multi-Label Prompting (AMu- LaP), a simple yet effective method to tackle the label …

Content truncated to 500 words