LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusion

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 14165–14178 July 9-14, 2023 ©2023 Association for Computational Linguistics LLM-BLE NDER: Ensembling Large Language Models with Pairwise Ranking and Generative Fusion Dongfu Jiang∑ Xiang Ren∫π Bill Yuchen Linπ dongfu@zju.edu.cn, xiangren@usc.edu, yuchenl@allenai.org πAllen Institute for Artificial Intelligence ∫University of Southern California ∑Zhejiang University Abstract We present LLM-BLENDER, an ensembling framework designed to attain consistently su- perior performance by leveraging the diverse strengths of multiple open-source large lan- guage models (LLMs). Our framework con- sists of two modules: PAIRRANKER and GEN- FUSER, addressing the observation that opti- mal LLMs for different examples can signif- icantly vary. PAIRRANKER employs a spe- cialized pairwise comparison method to dis- tinguish subtle differences between candidate outputs. It jointly encodes the input text and a pair of candidates, using cross-attention en- coders to determine the superior one. Our re- sults demonstrate that PAIRRANKER exhibits the highest correlation with ChatGPT-based ranking. Then, GENFUSER aims to merge the top-ranked candidates, generating an improved output by capitalizing on their strengths and mitigating their weaknesses. To facilitate large- scale evaluation, we introduce a benchmark dataset, MixInstruct, which is a mixture of multiple instruction datasets featuring oracle pairwise comparisons. Our LLM-BLEND E R significantly outperform individual LLMs and baseline methods across various metrics, estab- lishing a substantial performance gap. 1 2 1 Introduction Large language models (LLMs) have shown im- pressive performance in diverse tasks, primarily due to their capacity to follow instructions and ac- cess extensive, high-quality data, showing a promis- ing future for artificial general intelligence (Bubeck et al., 2023). However, prominent LLMs such as GPT-4 and PaLM (Chowdhery et al., 2022) are closed-source, restricting insights into their archi- tectures and training data. Open-source LLMs like 1https://yuchenlin.xyz/LLM-Blender 2The experiments on summarization, translation, and con- strained generation tasks in the prior version have been moved to the appendix. Instead, we mainly present our work in the context of instruction-following data and LLMs in this version. Open Assistant 12.61% Koala 6.71% Alpaca 11.61% Baize 11.61% StableLM 1.90% FLAN-T5 0.80% Vicuna 21.22% Dolly V2 4.50% MOSS 12.91% ChatGLM 8.51% MPT 7.61% Percentage of Examples Where Each Model Ranks First Which LLM should I use for my input? All! I can ensemble! Figure 1: Motivation of ensembling LLMs. Based on this pie chart about the percentage of examples where each LLM ranks 1st, we can see that optimal LLMs for different examples can significantly vary. Pythia (Biderman et al., 2023), LLaMA (Touvron et al., 2023), and Flan-T5 (Chung et al., 2022) of- fer a chance to fine-tune these models on custom instruction datasets, enabling the development of smaller yet efficient LLMs, such as Alpaca, Vi- cuna (Chiang et al., 2023), OpenAssistant (LAION- AI, 2023), and MPT (MosaicML, 2023). The open-source LLMs exhibit diverse strengths and weaknesses due to variations in data, archi- tectures, and hyperparameters, making them com- plementary to each other. Figure 1 illustrates the distribution of best LLMs on 5,000 instructions that we collected. More …

Content truncated to 500 words