Towards more variation in text generation: Developing and evaluating variation models for choice of referential form

Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pages 568–577, Berlin, Germany, August 7-12, 2016. c⃝2016 Association for Computational Linguistics Towards more variation in text generation: Developing and evaluating variation models for choice of referential form Thiago Castro Ferreira and Emiel Krahmer and Sander Wubben Tilburg center for Cognition and Communication (TiCC) Tilburg University The Netherlands {tcastrof,e.j.krahmer,s.wubben}@tilburguniversity.edu Abstract In this study, we introduce a non- deterministic method for referring expres- sion generation. We describe two models that account for individual variation in the choice of referential form in automatically generated text: a Naive Bayes model and a Recurrent Neural Network. Both are eval- uated using the VaREG corpus. Then we select the best performing model to gen- erate referential forms in texts from the GREC-2.0 corpus and conduct an evalu- ation experiment in which humans judge the coherence and comprehensibility of the generated texts, comparing them both with the original references and those pro- duced by a random baseline model. 1 Introduction Automatic text generation is the process of con- verting non-linguistic data into coherent and com- prehensible text (Reiter and Dale, 2000). In recent years, interest in text generation has substantially increased, due to the emergence of new applica- tions such as “robot-journalism” (Clerwall, 2014). Even though computers these days are perfectly capable of automatically producing text, the re- sults are arguably often rather rigid, always pro- ducing the same kind and style of text, which makes them somewhat “boring” to read, especially when reading multiple texts in succession. Human-written texts, by contrast, do not suf- fer from this problem, presumably because hu- man authors have an innate tendency to produce variation in their use of words and constructions. Indeed, psycholinguistic research has shown that when speakers produce referring expressions in comparable contexts, they non-deterministically vary both the form and the contents of their refer- ences (Dale and Viethen, 2010; Van Deemter et al., 2012). In this paper, we present and evaluate mod- els of referring expression generation that mimic this human non-determinacy and show that this enables us to generate varied references in texts, which, in terms of coherence and comprehensi- bility, did not yield significant differences from human-produced references according to human judges. In particular, in this study we focus on the choice of referential form, which is the first de- cision to be made by referring expression gener- ation models (Reiter and Dale, 2000) and which determines whether a reference takes the form of a proper name, a pronoun, a definite description, etc. Several such models have been proposed (Reiter and Dale, 2000; Henschel et al., 2000; Callaway and Lester, 2002; Krahmer and Theune, 2002; Gupta and Bandopadhyay, 2009; Greenbacker and McCoy, 2009). However, all of these are fully de- terministic, always choosing the same referential form in the same context. The fact that these models are generally based on text corpora which have only one gold standard form per reference (the one produced by the orig- inal author) does not help …

Content truncated to 500 words