Annotating Discourse Relations in Spoken Language: A Comparison of the PDTB and CCR Frameworks

Annotating Discourse Relations in Spoken Language: A Comparison of the PDTB and CCR Frameworks Ines Rehbein, Merel Scholman, Vera Demberg Saarland University Campus C7.4, 66123, Saarbr¨ucken {rehbein, m.c.j.scholman, vera}@coli.uni-saarland.de Abstract In discourse relation annotation, there is currently a variety of different frameworks being used, and most of them have been developed and employed mostly on written data. This raises a number of questions regarding interoperability of discourse relation annotation schemes, as well as regarding differences in discourse annotation for written vs. spoken domains. In this paper, we describe our work on annotating two spoken domains from the SPICE Ireland corpus (telephone conversations and broadcast interviews) according to two different discourse annotation schemes, PDTB 3.0 and CCR. We show that annotations in the two schemes can largely be mapped onto one another, and discuss differences in operationalisations of discourse relation schemes which present a challenge to automatic mapping. We also observe systematic differences in the prevalence of implicit discourse relations in spoken data compared to written texts, and find that there are also differences in the types of causal relations between the domains. Finally, we find that PDTB 3.0 addresses many shortcomings of PDTB 2.0 wrt. the annotation of spoken discourse, and suggest further extensions. The new corpus has roughly the size of the CoNLL 2015 Shared Task test set, and we hence hope that it will be a valuable resource for the evaluation of automatic discourse relation labellers. Keywords: Annotation of discourse relations (DRs), interoperability of annotation schemes, DRs in spoken and written genres 1. Introduction Over the last decade, research in NLP has widened its scope, moving beyond the sentence level to analysing the discourse structure of a text. This has resulted in the cre- ation of discourse-annotated corpora, such as the Penn Dis- course Treebank (PDTB) (Prasad et al., 2008), the Rhetori- cal Structure Theory (RST) Treebank (Carlson et al., 2002), and the Annodis corpus (Segmented Discourse Represen- tation Theory, SDRT) (Afantenos et al., 2012), to name a few. However, as of yet, the different frameworks are not inter-operable, nor is there a unified scheme for dis- course annotation (but see the proposals by Benamara and Taboada (2015), Chiarcos (2014), Bunt et al. (2012), and Sanders et al. (In preparation)). Most discourse-annotated corpora are based on written rather than spoken text. This point is crucial, as spoken and written texts are produced and processed differently (Cuenca, 2015): Spoken communication is characterised by a high degree of interactivity that requires turn-taking devices for discourse management; sentence length on av- erage is shorter, and the pressure of rapid online processing often leads to disfluent structures. In contrast to written communication, the speaker and the hearer have access to additional channels of communication, such as visual in- formation or, at least, audio cues such as pitch and sentence stress, and we observe many elliptical structures and omis- sions. We expect that these differences will be reflected in the use of discourse relations (DRs) in the spoken domain. The above considerations raise …

Content truncated to 500 words