Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 2717–2739 July 9-14, 2023 ©2023 Association for Computational Linguistics Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters Boshi Wang1 Sewon Min2 Xiang Deng1 Jiaming Shen3 You Wu3 Luke Zettlemoyer2 Huan Sun1 1The Ohio State University 2University of Washington {wang.13930,deng.595,sun.397}@osu.edu3Google Research {sewon,lsz}@cs.washington.edu, {jmshen,wuyou}@google.com Abstract Chain-of-Thought (CoT) prompting can dra- matically improve the multi-step reasoning abil- ities of large language models (LLMs). CoT explicitly encourages the LLM to generate in- termediate rationales for solving a problem, by providing a series of reasoning steps in the demonstrations. Despite its success, there is still little understanding of what makes CoT prompting effective and which aspects of the demonstrated reasoning steps contribute to its performance. In this paper, we show that CoT reasoning is possible even with invalid demonstrations—prompting with invalid rea- soning steps can achieve over 80-90% of the performance obtained using CoT under various metrics, while still generating coherent lines of reasoning during inference. Further experi- ments show that other aspects of the rationales, such as being relevant to the query and correctly ordering the reasoning steps, are much more important for effective CoT reasoning. Overall, these findings both deepen our understanding of CoT prompting, and open up new questions regarding LLMs’ capability to learn to reason in context.1 1 Introduction Large language models (LLMs) can perform new tasks during inference when prompted with a few demonstrations (Brown et al., 2020). Chain-of- Thought (CoT) prompting (Wei et al., 2022) can (Figure 1) improve the ability of sufficiently large LLMs to do complex and multi-step reasoning. In addition to (query, answer) example-pair demon- strations, CoT prompting includes a rationale (col- ored part in Figure 1) for each example, i.e., a series of reasoning steps towards the answer, which en- courages the LLM to explicitly generate its interme- diate reasoning process before predicting the final answer. Despite its successes, there is little under- standing of what makes CoT prompting effective 1Our code and model input/output are available here. In-context Demonstration Inference by LLM Query Leah had 32 chocolates and her sister had 42. If they ate 35, how many pieces do they have left in total? Julie is reading a 120-page book. Yesterday, she was able to read 12 pages and today, she read twice as many pages as yesterday. If she wants to read half of the remaining pages tomorrow, how many pages should she read? Standard The answer is 39. The answer is 18. ✗ CoT Originally, Leah had 32 chocolates and her sister had 42. So in total they had 32 + 42 = 74. After eating 35, they had 74 - 35 = 39 pieces left in total. The answer is 39. Julie is reading a 120-page book. Yesterday, she read 12 pages and today, she read 24 pages. So she read a total of 12 + 24 = 36 pages. Now she has 120 - 36 = 84 pages left. Since …

Content truncated to 500 words