Individual Variation in the Choice of Referential Form

Proceedings of NAACL-HLT 2016, pages 423–427, San Diego, California, June 12-17, 2016. c⃝2016 Association for Computational Linguistics Individual Variation in the Choice of Referential Form Thiago Castro Ferreira and Emiel Krahmer and Sander Wubben Tilburg center for Cognition and Communication (TiCC) Tilburg University The Netherlands {tcastrof,e.j.krahmer,s.wubben}@uvt.nl Abstract This study aims to measure the variation be- tween writers in their choices of referential form by collecting and analysing a new and publicly available corpus of referring expres- sions. The corpus is composed of referring expressions produced by different participants in identical situations. Results, measured in terms of normalized entropy, reveal substan- tial individual variation. We discuss the prob- lems and prospects of this finding for auto- matic text generation applications. 1 Introduction Automatic text generation is the process of automat- ically converting data into coherent text - practical applications range from weather reports (Goldberg et al., 1994) to neonatal intensive care reports (Portet et al., 2009). One important way to achieve co- herence in texts is by generating appropriate refer- ring expressions throughout the text (Krahmer and van Deemter, 2012). In this generation process, the choice of referential form is a crucial task (Re- iter and Dale, 2000): when referring to a person or object in a text, should the system use a proper name (“Phillip Anschutz”), a definite description (“the American entrepreneur”) or a pronoun (“he”)? Despite the large amount of algorithms developed for deciding upon the form of a referring expression (Callaway and Lester, 2002; Greenbacker and Mc- Coy, 2009; Gupta and Bandopadhyay, 2009; Or˘asan and Dornescu, 2009; Greenbacker et al., 2010), it is difficult to know how well these algorithms actu- ally perform. Typically, such algorithms are eval- uated against a corpus of human written texts, pre- dicting what form each reference should have in a given context. Now consider a situation in which the algorithm predicts that a reference should be a description, while this same reference is a pronoun in the corpus text. Should this count as an error? The answer is: it depends. The use of a pronoun does not necessarily mean that the use of a descrip- tion is incorrect. In fact, other writers might have used a description as well. In general, corpora of referring expressions have only one gold standard referential form for each sit- uation, while different writers may conceivably vary in the referential form they would use. This compli- cates the development and evaluation of text gener- ation algorithms, since these will typically attempt to predict the corpus gold standard, which may not always be representative of the choices of different writers. Although recent work in text generation has explored individual variation in the content deter- mination of definite descriptions (Viethen and Dale, 2010; Ferreira and Paraboni, 2014), to the best of our knowledge this has not been systematically ex- plored for choosing referential forms. In this paper, we collect and analyze a new cor- pus to address this issue. In the collection, we pre- sented different writers with texts …

Content truncated to 500 words