Bounding the Capabilities of Large Language Models in Open Text Generation with Prompt Constraints

Findings of the Association for Computational Linguistics: EACL 2023, pages 1982–2008 May 2-6, 2023 ©2023 Association for Computational Linguistics Bounding the Capabilities of Large Language Models in Open Text Generation with Prompt Constraints Albert Lu∗, Hongxin Zhang∗1, Yanzhe Zhang, Xuezhi Wang2, Diyi Yang3 Georgia Institute of Technology, 1Shanghai Jiao Tong University, 2Google, 3Stanford University {albert.lu, z_yanzhe}@gatech.edu, 1icefox@sjtu.edu.cn 2xuezhiw@google.com, 3diyiy@cs.stanford.edu Abstract The limits of open-ended generative models are unclear, yet increasingly important. What causes them to succeed and what causes them to fail? In this paper, we take a prompt- centric approach to analyzing and bounding the abilities of open-ended generative models. We present a generic methodology of anal- ysis with two challenging prompt constraint types: structural and stylistic. These con- straint types are categorized into a set of well- defined constraints that are analyzable by a single prompt. We then systematically cre- ate a diverse set of simple, natural, and use- ful prompts to robustly analyze each individual constraint. Using the GPT-3 text-davinci-002 model as a case study, we generate outputs from our collection of prompts and analyze the model’s generative failures. We also show the generalizability of our proposed method on other large models like BLOOM and OPT. Our results and our in-context mitigation strate- gies reveal open challenges for future research. We have publicly released our code at https: //github.com/SALT-NLP/Bound-Cap-LLM. 1 Introduction The recent success of large language models (LLM) (Brown et al., 2020; Devlin et al., 2018; Raffel et al., 2019) has transformed the field of natural language processing (NLP). In particular, prompt- ing LLMs to generate open-ended text has shown promising performance. The existing and potential applications of open-ended text generation are far- reaching, spanning domains such as QA (Zhu et al., 2021), story generation (Fan et al., 2018), code generation (Chen et al., 2021a), human-assisted creativity (Akoury et al., 2020), open-ended dia- logue (Zhang et al., 2020), and the varied usages of ChatGPT 1. However, as LLMs continue to rise, ∗Equal contribution. 1https://chat.openai.com Figure 1: Example Model Failures: These two prompts are from our taxonomy and the two responses are generated by GPT-3. There are clear deficiencies that are described further in this paper. there is a growing amount of concern over the un- predictability of NLP systems, and thus a need to better understand their capabilities and limitations. An extensive analysis of open-ended text genera- tion is imperative to understand their capabilities, limitations, and areas for improvement. Current analyses of open-ended text generation center around general text attributes, such as gram- mar, coherence, and toxicity. These analyses are used to understand general aspects of model gener- ations, but they do not analyze model performance in regards to the prompt. The next step in this field is to analyze prompt-specific performance by break- ing down the vast space of open text generation into a taxonomy of simple, natural, and useful prompts. A fine-grained understanding of what prompts a model can and can’t handle creates clear bounds on model capabilities, and drives model explainability and future …

Content truncated to 500 words