metaTED: a Corpus of Metadiscourse for Spoken Language

metaTED: a Corpus of Metadiscourse for Spoken Language Rui Correia1,2, Nuno Mamede2, Jorge Baptista2,3, Maxine Eskenazi1 1 LTI - Carnegie Mellon University, Pittsburgh, USA 2 INESC-ID, Lisbon, Portugal 3 Universidade do Algarve, Faro, Portugal rcorreia@cs.cmu.edu, Nuno.Mamede@inesc-id.pt, jbaptis@ualg.pt, max@cs.cmu.edu Abstract This paper describes metaTED – a freely available corpus of metadiscursive acts in spoken language collected via crowdsourcing. Metadiscursive acts were annotated on a set of 180 randomly chosen TED talks in English, spanning over different speakers and topics. The taxonomy used for annotation is composed of 16 categories, adapted from ¨Adel (2010). This adaptation takes into account both the material to annotate and the setting in which the annotation task is performed. The crowdsourcing setup is described, including considerations regarding training and quality control. The collected data is evaluated in terms of quantity of occurrences, inter-annotator agreement, and annotation related measures (such as average time on task and self-reported confidence). Results show different levels of agreement among metadiscourse acts (α ∈ [0.15; 0.49]). To further assess the collected material, a subset of the annotations was submitted to expert appreciation, who validated which of the marked occurrences truly correspond to instances of the metadiscursive act at hand. Similarly to what happened with the crowd, experts revealed different levels of agreement between categories (α ∈ [0.18; 0.72]). The paper concludes with a discussion on the applicability of metaTED with respect to each of the 16 categories of metadiscourse. Keywords: metadiscourse, spoken language, crowdsourcing 1. Introduction Commonly referred to as discourse about discourse, metadiscourse is composed of rhetorical acts and patterns used to make the discourse structure explicit, acting as a way to guide the audience. Crismore et al. (1993) de- fine metadiscourse as “linguistic material in texts, writ- ten or spoken, which does not add anything to the propo- sitional content but that is intended to help the listener or reader organize, interpret and evaluate the information given”. Some examples of metadiscursive acts include in- troductions (I’m going to talk about. . .; In this paper we present. . .), conclusions (In sum,. . .), or emphasis (The take-home message. . .). This study focuses on the function of metadiscourse in spo- ken communication. As previously mentioned, metadis- course reflects the explicit intention of the speaker and, therefore, its analysis uncovers a map of explicitly stated discourse functions. This idea contrasts with implicit infor- mation conveyed, for instance, by means of prosody. For example, the speaker is able to enumerate simply by means of prosody, such as in the sentence “they use several data modalities (localization, physiological, ...)”, where the speaker will resort to pauses and intonation patterns commonly associated with the function of enumerating. On the other hand, if the items in an enumeration are com- plex and/or more important to the content, the speaker may explicitly signal them such as in “They use several data modalities. The first, localization is ...The second is physiological . . . ”. Another example of the implicit vs explicit use of discourse functions is …

Content truncated to 500 words