Data-to-text Generation by Splicing Together Nearest Neighbors

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 4283–4299 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 4283 Data-to-text Generation by Splicing Together Nearest Neighbors Sam Wiseman∗ Duke University swiseman@cs.duke.edu Arturs Backurs Microsoft Research arturs.backurs@microsoft.com Karl Stratos Rutgers University karl.stratos@rutgers.edu Abstract We propose to tackle data-to-text generation tasks by directly splicing together retrieved segments of text from “neighbor” source- target pairs. Unlike recent work that condi- tions on retrieved neighbors but generates text token-by-token, left-to-right, we learn a policy that directly manipulates segments of neigh- bor text, by inserting or replacing them in par- tially constructed generations. Standard tech- niques for training such a policy require an oracle derivation for each generation, and we prove that finding the shortest such derivation can be reduced to parsing under a particular weighted context-free grammar. We find that policies learned in this way perform on par with strong baselines in terms of automatic and human evaluation, but allow for more inter- pretable and controllable generation. 1 Introduction There has been recent interest in text generation sys- tems that make use of retrieved “neighbors” — ex- amples of good text retrieved from a database, per- haps paired with the source information on which these example texts condition — on the hope that these neighbors might make a generation task eas- ier, or the system more interpretable or control- lable (Song et al., 2016; Weston et al., 2018; Guu et al., 2018; Zhang et al., 2018; Peng et al., 2019, inter alia). Whereas most work along these lines has adopted a conventional encoder-decoder approach, conditioning on the retrieved neighbors and then au- toregressively generating text token-by-token from left to right, we instead propose to generate text by directly splicing together segments of text from retrieved neighbors. Generating in this way aligns with the intuition that in settings such as data-to- text generation it ought to be sufficient to retrieve ∗This work was done primarily while the first two authors were at TTI-Chicago. sentences similar to the one that must be gener- ated, and then merely change some details, such as names or dates. There are notable advantages to a generation-by- splicing approach. First, generation becomes more interpretable: it is always clear from which neigh- bor a particular piece of generated text derives, and it is also clear how these pieces have come together to form the generated text. Generation-by-splicing may also increase our control over the generated text, and we suspect that approaches that make clear the provenance of each piece of generated text (as ours does) will be useful in preventing text generation systems from emitting harmful or bi- ased text (Sheng et al., 2019; Wallace et al., 2019; Gehman et al., 2020, inter alia). That is, we might imagine preventing systems from emitting harmful or biased text by only allowing generation from approved neighbor examples. Methodologically, we implement this generation- by-splicing approach by training a policy to directly insert or replace spans of neighbor text at arbitrary positions …

Content truncated to 500 words