Iteratively Prompt Pre-trained Language Models for Chain of Thought

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 2714 - 2730 December 7-11, 2022 ©2022 Association for Computational Linguistics Iteratively Prompt Pre-trained Language Models for Chain of Thought Boshi Wang, Xiang Deng and Huan Sun The Ohio State University, Columbus, OH {wang.13930,deng.595,sun.397}@osu.edu Abstract While Pre-trained Language Models (PLMs) internalize a great amount of world knowl- edge, they have been shown incapable of re- calling these knowledge to solve tasks requir- ing complex & multi-step reasoning. Similar to how humans develop a “chain of thought” for these tasks, how can we equip PLMs with such abilities? In this work, we explore an iterative prompting framework, a new prompt- ing paradigm which progressively elicits rele- vant knowledge from PLMs for multi-step in- ference. We identify key limitations of existing prompting methods, namely they are either re- stricted to queries with a single identifiable re- lation/predicate, or being agnostic to input con- texts, which makes it difficult to capture vari- abilities across different inference steps. We propose an iterative context-aware prompter, which addresses these limitations by learning to dynamically synthesize prompts conditioned on the current step’s contexts. Experiments on three datasets involving multi-step reasoning show the effectiveness of the iterative scheme and the context-aware prompter design.1 1 Introduction Humans can develop a “chain of thought” for com- plex decision making. For example, when asked the question (Q) shown in Figure 1, which involves composition, an important type of multi-step rea- soning, humans apply two consecutive steps to de- rive the final answer: 1) find “father” of the topic entity “Gwilym Lloyd George” (C1); 2) find “birth- place” of the entity returned in the first step (C2). Recently, large-scale pre-trained language mod- els (PLMs) have been shown capable of internal- izing a great amount of simple factual knowledge such as C1 and C2, yielding competitive perfor- mance on a range of knowledge-intensive tasks without resorting to any external knowledge source 1Our source code is available at https://github. com/sunlab-osu/IterPrompt. Iterative Prompting Standard Probing C1: “David Lloyd George is the father of Gwilym Lloyd George” Q: “What is the place of birth of Gwilym Lloyd George’s father?” (Answer: Manchester) PLM “Who is the father of Gwilym Lloyd George?” “David Lloyd George” “Unknown” Q Prompter Q Q & C1 Prompter C2: “Manchester is the place of birth of David Lloyd George” Figure 1: Our Iterative Prompting approach (on the right), compared with Standard Probing (on the left). In Standard Probing, a question is directly fed to the PLM to output the final answer, which could work for simple factual questions but fails for complex questions that require multi-step reasoning. In contrast, we augment the PLM with a Prompter, which learns to iteratively prompt the PLM to recall a series of knowledge and derive a “chain of thought”. (Petroni et al., 2019; Shin et al., 2020; Zhong et al., 2021; Roberts et al., 2020; Lee et al., 2020). How- ever, work such as (Talmor et al., 2020a; Kassner et al., 2020; Rae et …

Content truncated to 500 words