Persian Discourse Treebank and coreference corpus

Persian Discourse Treebank and coreference corpus Azadeh Mirzaei, Pegah Safari Department of Linguistics, Allameh Tabataba'i University, Tehran, Iran; Department of Technology, Alzahra University, Tehran, Iran azadeh.mirzaei@atu.ac.ir, phsafari@yahoo.com Abstract This research addresses the investigation of intra-document relations based on two major approaches: discourse analysis and coreference resolution which results in building the first Persian discourse Treebank and a comprehensive Persian coreference corpus. In discourse analysis, we have explored sentence-level relations defined between clauses in complex sentences. So we specified 34682 discourse relations, the sense of the relations, their arguments and their attributes mainly consisted of the source of the message and its type. Our discourse analysis is based on a corpus consisted of 30000 individual sentences with morphologic, syntactic and semantic labels and nearly half a million tokens. Also 18336 of these sentences are double-annotated. For coreference annotation, since a document-based corpus was needed, we prepared a new corpus consisted of 547 documents and 212646 tokens which is still under development. We enriched it with morphological and syntactical labels and added coreference information at the top. Currently, we have annotated 6511 coreference chains and 21303 mentions with a comprehensive annotation scheme to compensate some specification of Persian such as being pro-drop or lacking gender agreement information. Keywords: Persian discourse Treebank, Persian Coreference Corpus, logical relation, mention, referent. 1. Introduction In this research we addressed the investigation of intra- document relations with two major approaches: discourse analysis and co-reference resolution. In discourse analysis, we can inspect the logical relations between sentences inside a document and also we can explore sentence-level relations defined between clauses in complex sentences. Since there was already a rich annotated Persian corpus consisted of 30000 individual sentences, we augmented sentence-level discourse information on the top of its morphologic, syntactic and semantic layers. But for coreference annotation, a document-based corpus was needed; because in this annotation, the relation between a referent and its mention is defined inside a document and it is not anymore restricted to an individual sentence. So we prepared a new corpus and enriched it with morphological and syntactical labels to be used for learning processes in the future. Then we added coreference information at the top. Currently, we are adding a new document-level discourse annotation to our new corpus as well. So in this paper, first of all, we describe the process of preparing the first Persian discourse Treebank and then elaborate the preparation process of our first comprehensive Persian coreference corpus. 2. Persian Discourse Treebank Persian Discourse Treebank (PerDTB) as the first discourse corpus in Persian, has been developed based on the schema of Penn Discourse Treebank (weber et al., 2003; Prasad et al., 2008) which has been used in discourse projects of the other languages such as Arabic (Al-Saif and Markert, 2010), Chinese (Zhou and Xue, 2012), Czech (Mladov´a et al., 2008), Hindi (Oza et al., 2009), Italian (Tonelli et al., 2010) and Turkish (Zeyrek and Webber, 2008). The corpus is based on nearly 30,000 sentences which has received morphologic and syntactic labels …

Content truncated to 500 words