QUD-Based Annotation of Discourse Structure and Information Structure: Tool and Evaluation

QUD-Based Annotation of Discourse Structure and Information Structure: Tool and Evaluation Kordula De Kuthy∗, Nils Reiter†, Arndt Riester† ∗Department of Linguistics, Wilhelmstr. 19, 72074 T¨ubingen, Germany †Institute for Natural Language Processing, Pfaffenwaldring 5B, 70569 Stuttgart, Germany kdk@sfs.uni-tuebingen.de,{nils.reiter,arndt.riester}@ims.uni-stuttgart.de Abstract We discuss and evaluate a new annotation scheme and discourse-analytic method, the QUD-tree framework. We present an annotation study, in which the framework, based on the concept of Questions under Discussion, is applied to English and German interview data, using TreeAnno, an annotation tool specially developed for this new kind of discourse annotation. The results of an inter-annotator agreement study show that the new annotation method allows for reasonable agreement with regard to discourse structure and good agree- ment with regard to the annotation of information structure, which covers focus, background, contrastive topic and non-at-issue material. Keywords: discourse structure, information structure, annotation, inter-annotator agreement, Question under Discussion 1. Introduction In this paper, we evaluate a new annotation scheme and discourse-analytic method, the QUD-tree framework, de- veloped in Reyle and Riester (2016), Riester (to appear) and Riester et al. (to appear). Its purpose is the cross-linguistic analysis of information structure and discourse structure of textual data. We furthermore introduce a new tool, TreeAnno, which enables the analyst to semi-automatically segment texts, systematically enhance them with implicit Questions under Discussion (QUDs), and transform the data into a new kind of discourse tree called QUD tree. 2. QUD trees For several decades scholars have been claiming that im- plicit questions (so-called Questions under Discussion, or QUDs) are constitutive of the internal structure of texts, e.g. Polanyi (1988), Stutterheim and Klein (1989), van Kup- pevelt (1995) or, recently, e.g. Onea (2016), Velleman and Beaver (2016), Riester (to appear). This means that ev- ery statement contained in a text is seen as the immediate answer to precisely one implicit or explicit QUD, and po- tentially also as an indirect answer to one or several more general QUDs. QUDs can be thought of as fine-grained, silent headlines of sections, subsections etc., down to the bottom-level of atomic assertions. The content of the lat- ter bottom-level QUDs is, at the same time, the background (or topic) of the assertions dominated by it, while the con- stituent of the assertion that answers the question is called the focus. Therefore, QUDs also determine the information structure of the assertions contained in the text (Roberts, 2012), which is then reflected in specific choices of con- stituent order or the use of cleft constructions and, as to spoken discourse, in a characteristic prosodic realization. To our knowledge, the QUD-tree method (Riester et al., to appear), briefly sketched in the following, is the first cor- pus annotation framework which actually implements the reconstruction of QUDs, which have, so far, mainly been discussed from a theoretical or experimental perspective. It is at the same time one of the few frameworks that deals with discourse structure and information structure simulta- neously; but compare e.g. Kruijff-Korbayov´a and Steedman (2003), Polanyi et al. (2003), Pol´akov´a et al. (2013). …

Content truncated to 500 words