Simple and Effective Retrieve-Edit-Rerank Text Generation

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 2532–2538 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 2532 Simple and Effective Retrieve-Edit-Rerank Text Generation Nabil Hossain† Marjan Ghazvininejad♠ †Dept. Computer Science, University of Rochester ♠Facebook AI Research nhossain@cs.rochester.edu, {ghazvini,lsz}@fb.com Luke Zettlemoyer♠ Abstract Retrieve-and-edit seq2seq methods typically retrieve an output from the training set and learn a model to edit it to produce the final out- put. We propose to extend this framework with a simple and effective post-generation ranking approach. Our framework (i) retrieves sev- eral potentially relevant outputs for each in- put, (ii) edits each candidate independently, and (iii) re-ranks the edited candidates to se- lect the final output. We use a standard editing model with simple task-specific re- ranking approaches, and we show empirically that this approach outperforms existing, sig- nificantly more complex methodologies. Ex- periments on two machine translation (MT) datasets show new state-of-art results. We also achieve near state-of-art performance on the Gigaword summarization dataset, where our analyses show that there is significant room for performance improvement with better can- didate output selection in future work. 1 Introduction Retrieve-and-edit text generation methods have re- ceived significant recent interest; editing human- authored text can potentially avoid many of the challenges that are seen while generating text from scratch, including the tendency to be overly repeti- tive or to degrade on longer texts (Holtzman et al., 2018, 2019). Retrieve-and-edit methods have been developed for summarization (Cao et al., 2018), machine translation (Wu et al., 2019), language modeling (Guu et al., 2018), and conversation gen- eration (Weston et al., 2018). These methods first retrieve a single output from the training set, and then use a learned model to edit it into the final output. In this paper, we show that generation perfor- mance can be improved with a retrieve-edit-rerank approach that instead retrieves a set of outputs from Figure 1: Our retrieve-edit-rerank framework, generat- ing candidate outputs with three retrieved outputs, and re-ranking ˆy2 as the best candidate post-generation. the training set, edits each independently, and then re-ranks the results to produce the final output. Fig- ure 1 shows an overview of the approach. We use standard keyword-based retrieval and a simple editor, where the retrieved output is concate- nated to the original input to train a Transformer- based seq2seq editing model. Our final re-ranking step is task specific, but again very simple in every case. Our goal here is not to find the best possible way to do the re-ranking. Instead, we show that gains are possible and that it helps to see what edits are made for multiple candidates before making the final decision, instead of following previous work by trying to select a single candidate before editing. We evaluate performance on the Gigaword sum- marization dataset (Rush et al., 2015) and on the English to Dutch (EN-NL) and the English to Hun- garian (EN-HU) machine translation (MT) tasks, following Bulte and Tezcan (2019). For MT, we ex- perimented with …

Content truncated to 500 words