Verify-and-Edit: A Knowledge-Enhanced Chain-of-Thought Framework

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 5823–5840 July 9-14, 2023 ©2023 Association for Computational Linguistics Verify-and-Edit: A Knowledge-Enhanced Chain-of-Thought Framework Ruochen Zhao 1∗ Xingxuan Li 1,2∗† Shafiq Joty 1,3‡ Chengwei Qin 1 Lidong Bing 2 1 Nanyang Technological University, Singapore 2 DAMO Academy, Alibaba Group 3 Salesforce AI {ruochen002, chengwei003}@e.ntu.edu.sg {xingxuan.li, l.bing}@alibaba-inc.com srjoty@ntu.edu.sg Abstract As large language models (LLMs) have be- come the norm in NLP, demonstrating good performance in generation and reasoning tasks, one of its most fatal disadvantages is the lack of factual correctness. Generating unfactual texts not only leads to lower performances but also degrades the trust and validity of their appli- cations. Chain-of-Thought (CoT) prompting improves trust and model performance on com- plex reasoning tasks by generating interpretable reasoning chains, but still suffers from factu- ality concerns in knowledge-intensive tasks. In this paper, we propose the Verify-and-Edit framework for CoT prompting, which seeks to increase prediction factuality by post-editing reasoning chains according to external knowl- edge. Building on top of GPT-3, our frame- work lead to accuracy improvements in mul- tiple open-domain question-answering tasks. For reproducing our results and extending the framework further, we make our codebase avail- able at https://github.com/RuochenZhao/Verify- and-Edit 1 Introduction Large Language Models (LLMs) have become the new norm in many downstream NLP tasks. In utilizing these LLMs, Chain-of-Thought (CoT) prompting (Wei et al., 2022) is found to improve performances for tasks that require complex reason- ing, such as math word problems, commonsense reasoning, and symbolic manipulation. At the same time, it is able to generate interpretable reasoning chains. Recent work further explored how to use these reasoning chains to select better predictions. However, the primary focus of these methods has been to improve end-task performance by utiliz- ing generated CoTs as-is. For example, Ye and ∗Equal contribution. †Xingxuan Li is under the Joint Ph.D. Program between Alibaba and Nanyang Technological University. ‡Work done when the author was on leave from NTU. Standard Question Chain-of-thought Self-Consistency: less than majority agree Verify External Knowledge Retrieval Edit Rationales New Prediction Of all the teams John Nyskohus played for, which team was known as "the Black and Whites?" Newcastle United. First, John Nyskohus played for the Norwegian football team Odd Grenland. Second, Odd Grenland is known as "the Black and Whites." The answer is Odd Grenland. What team did John Nyskohus play for? What team is known as "the Black and Whites?" John Nyskohus ... is an Australian former soccer player who played club football for USC Lion ... and Adelaide City in the National Soccer League ... Adelaide City Football Club is an Australian football (soccer) club based in Adelaide, South Australia. They are also known as "The Zebras" and "the Black and Whites. First, John Nyskohus played for Adelaide City in the National Soccer League. Second, Adelaide City Football Club is known as "the Black and Whites". The answer is Adelaide City Football Club. Figure 1: The Verify-and-Edit framework consists of five …

Content truncated to 500 words