Unnatural Instructions: Tuning Language Models with (Almost) No Human Labor

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 14409–14428 July 9-14, 2023 ©2023 Association for Computational Linguistics Unnatural Instructions: Tuning Language Models with (Almost) No Human Labor Or Honovichτ Thomas Scialomµ Omer Levyτµ Timo Schickµ τ Tel Aviv University µ Meta AI Abstract Instruction tuning enables pretrained language models to perform new tasks from inference- time natural language descriptions. These ap- proaches rely on vast amounts of human super- vision in the form of crowdsourced datasets or user interactions. In this work, we introduce Unnatural Instructions: a large dataset of cre- ative and diverse instructions, collected with virtually no human labor. We collect 64,000 examples by prompting a language model with three seed examples of instructions and elic- iting a fourth. This set is then expanded by prompting the model to rephrase each instruc- tion, creating a total of approximately 240,000 examples of instructions, inputs, and outputs. Experiments show that despite containing a fair amount of noise, training on Unnatural In- structions rivals the effectiveness of training on open-source manually-curated datasets, sur- passing the performance of models such as T0++ and Tk-Instruct across various bench- marks. These results demonstrate the poten- tial of model-generated data as a cost-effective alternative to crowdsourcing for dataset expan- sion and diversification. 1 Introduction Instruction tuning enables pretrained language models to generalize to unseen tasks in a zero-shot setting (Sanh et al., 2021; Wei et al., 2021). One way to collect examples of instructions and their execution is to reformulate existing NLP datasets in an explicit instruction-input-output format via prompt engineering (Mishra et al., 2022; Wang et al., 2022). However, the resulting data is limited to existing academic benchmarks, even though the instruction paradigm can describe any text-based task (Efrat and Levy, 2020). Alternatively, Ouyang et al. (2022) collect user-generated prompts and manually annotate their expected outputs, reflect- ing a different (and arguably more desirable) dis- tribution of the instruction space, but requiring a live application with existing users and major in- vestments in human annotation. Can we create a large dataset of instructions that is diverse in tasks, content, and phrasing, without human labor? We introduce Unnatural Instructions, a dataset of natural language instructions and their corre- sponding inputs and outputs. Inspired by recent work on utilizing language models for data genera- tion (Schick and Schütze, 2021b; Lee et al., 2021; Liu et al., 2022a), we collect data in a fully auto- matic manner by prompting a pretrained language model with three examples from the Super-Natural Instructions1 dataset (Mishra et al., 2022; Wang et al., 2022) and asking the model to generate a fourth (Figure 1). We repeat this process with 5 different seeds – i.e. the entire process requires only 15 instruction examples – to automatically pro- duce 64,000 diverse triplets of instructions, inputs, and outputs.2 We further diversify the dataset’s format by generating additional natural language paraphrases of each instruction, while preserving the contents of any input arguments and outputs, expanding the dataset …

Content truncated to 500 words