The Potsdam Commentary Corpus

The Potsdam Commentary Corpus Manfred Stede University of Potsdam Dept. of Linguistics Applied Computational Linguistics D-14415 Potsdam Germany stede@ling.uni-potsdam.de Abstract A corpus of German newspaper commentaries has been assembled and annotated with differ- ent information (and currently, to different de- grees): part-of-speech, syntax, rhetorical struc- ture, connectives, co-reference, and information structure. The paper explains the design deci- sions taken in the annotations, and describes a number of applications using this corpus with its multi-layer annotation. 1 Introduction A corpus of German newspaper commentaries has been assembled at Potsdam University, and annotated with different linguistic information, to different degrees. Two aspects of the corpus have been presented in previous papers ((Re- itter, Stede 2003) on underspecified rhetorical structure; (Stede 2003) on the perspective of knowledge-based summarization). This paper, however, provides a comprehensive overview of the data collection effort and its current state. At present, the ‘Potsdam Commentary Cor- pus’ (henceforth ‘PCC’ for short) consists of 170 commentaries from M¨arkische Allgemeine Zeitung, a German regional daily. The choice of the genre commentary resulted from the fact that an investigation of rhetorical structure, its interaction with other aspects of discourse structure, and the prospects for its automatic derivation are the key motivations for building up the corpus. Commentaries argue in favor of a specific point of view toward some polit- ical issue, often dicussing yet dismissing other points of view; therefore, they typically offer a more interesting rhetorical structure than, say, narrative text or other portions of newspapers. The choice of the particular newspaper was motivated by the fact that the language used in a regional daily is somewhat simpler than that of papers read nationwide. (Again, the goal of automatic analysis was responsible for this deci- sion.) This is manifest in the lexical choices but also in structural features. As an indication, in our core corpus, we found an average sentence length of 15.8 words and 1.8 verbs per sentence, whereas a randomly taken sample of ten com- mentaries from the national papers S¨uddeutsche Zeitung and Frankfurter Allgemeine has 19.6 words and 2.1 verbs per sentence. The com- mentaries in PCC are all of roughly the same length, ranging from 8 to 10 sentences. For il- lustration, an English translation of one of the commentaries is given in Figure 1. The paper is organized as follows: Section 2 explains the different layers of annotation that have been produced or are being produced. Sec- tion 3 discusses the applications that have been completed with PCC, or are under way, or are planned for the future. Section 4 draws some conclusions from the present state of the effort. 2 Layers of Annotation The corpus has been annotated with six differ- ent types of information, which are character- ized in the following subsections. Not all the layers have been produced for all the texts yet. There is a ‘core corpus’ of ten commentaries, for which the range of information (except for syntax) has been completed; the remaining data has been annotated to different degrees, as …

Content truncated to 500 words