Model Criticism for Long-Form Text Generation

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 11887 - 11912 December 7-11, 2022 ©2022 Association for Computational Linguistics Model Criticism for Long-Form Text Generation Yuntian Deng1, Volodymyr Kuleshov2, Alexander M. Rush2 1 Harvard University dengyuntian@seas.harvard.edu 2 Cornell University {kuleshov,arush}@cornell.edu Abstract Language models have demonstrated the abil- ity to generate highly fluent text; however, it remains unclear whether their output retains coherent high-level structure (e.g., story pro- gression). Here, we propose to apply a statis- tical tool, model criticism in latent space, to evaluate the high-level structure of the gener- ated text. Model criticism compares the dis- tributions between real and generated data in a latent space obtained according to an as- sumptive generative process. Different gener- ative processes identify specific failure modes of the underlying model. We perform experi- ments on three representative aspects of high- level discourse—coherence, coreference, and topicality—and find that transformer-based language models are able to capture topical structures but have a harder time maintaining structural coherence or modeling coreference. 1 Introduction It is now broadly accepted that neural language models can consistently generate fluent text (Rad- ford et al., 2019; Shoeybi et al., 2019; Brown et al., 2020; Smith et al., 2022). Yet, while large language models make few local word-level er- rors, human studies have shown that they still of- ten make “high-level” errors such as incoherence, self-contradictions, and off-topic generations (Dou et al., 2022). We hypothesize that researchers have focused on local fluency partly because it is easy to automatically evaluate through metrics such as perplexity and n-gram matching. Automatic as- sessment of high-level text generation quality has received less attention, partially because a single general-purpose metric does not exist. This work takes a step toward the automatic eval- uation of the high-level structure of the generated text by applying a tool from statistics, model criti- cism in latent space (Dey et al., 1998; Seth et al., 2019). Under this approach, we first project data to a latent space based on an assumptive generative process, and then compare the implied latent distri- butions between real data and language model sam- ples. This approach unifies past work for evaluating text generation under a single framework, includ- ing existing dimensionality reduction techniques such as probabilistic PCA (Wold et al., 1987), as well as previous applications of model criticism that were restricted to topic models (Mimno and Blei, 2011). By making different assumptions in the under- lying generative process, model criticism in latent space identifies specific failure modes of the gen- erated language. We demonstrate this on three representative high-level properties of the gener- ated discourse—coherence (Barzilay and Lapata, 2005), coreference (Chomsky, 1993), and topicality (Blei and Lafferty, 2006)—as well as on a synthetic dataset for which the true data generating process is known. Experiments using our proposed framework en- able us to make four observations about modern language models. First, we find that it is possible for a model to get strong word-level perplexity, yet fail to capture longer-term dynamics. …

Content truncated to 500 words