ARRAU: Linguistically-Motivated Annotation of Anaphoric Descriptions

ARRAU: Linguistically-Motivated Annotation of Anaphoric Descriptions Olga Uryupina1, Ron Artstein2, Antonella Bristot, Federica Cavicchio3, Kepa J. Rodriguez4, Massimo Poesio5,1 1Department of Information Engineering and Computer Science, University of Trento, 2Institute for Creative Technologies, University of Southern California, 3Sign Language Lab, University of Haifa, 4Archives Division, Yad Vashem, 5 School of Computer Science and Electronic Engineering, University of Essex uryupina@gmail.com, artstein@ict.usc.edu, lucanto137@libero.it, federica.cavicchio@gmail.com, kepa.rodriguez@yadvashem.org.il, poesio@essex.ac.uk Abstract This paper presents a second release of the ARRAU dataset: a multi-domain corpus with thorough linguistically motivated annotation of anaphora and related phenomena. Building upon the first release almost a decade ago, a considerable effort had been invested in improving the data both quantitatively and qualitatively. Thus, we have doubled the corpus size, expanded the selection of covered phenomena to include referentiality and genericity and designed and implemented a methodology for enforcing the consistency of the manual annotation. We believe that the new release of ARRAU provides a valuable material for ongoing research in complex cases of coreference as well as for a variety of related tasks. The corpus is publicly available through LDC. Keywords: Discourse, Anaphora, Coreference 1. Introduction Coreference resolution is a crucial step in deep text under- standing and as such is a vital prerequisite for a variety of high-level natural processing tasks, ranging from infor- mation extraction to summarization or machine translation. State-of-the-art statistical approaches to the task achieve ro- bust performance on relatively easy cases of coreference, especially since the vast model optimization efforts have been undertaken by various research groups for the recent SemEval and CoNLL coreference resolution tracks (Re- casens et al., 2010; Pradhan et al., 2011; Pradhan et al., 2012). More complex cases have been identified and in- vestigated since the first years of research on coreference resolution, however, they have been out of the scope of the mainstream community till very recently. One of the main reasons is the lack of appropriate datasets. Since its first release (Poesio and Artstein, 2008), the AR- RAU corpus has been used, on one hand, for research on more complex coreference phenomena, and, on the other hand, as a reference point for annotating coreference cor- pora in other languages. The current paper presents the second release of ARRAU. For the second release we have not only focused on augmenting the number of covered documents, but also invested a considerable effort into im- proving the data quality. This involved annotating more attributes and designing a methodology for cleaning up the annotations. The former allows to use ARRAU for a variety of coreference-related problems, for example, for identify- ing and sub-classifying non-referential expressions. This is especially crucial when we want to train models cover- ing complex cases of anaphora. The latter ensures the data quality and hence improves the performance of already ex- isting models through retraining on cleaner data. The rest of this paper is organized as follows. In the next section, we summarize the ARRAU annotation guidelines highlighting the differences between the two releases of ARRAU. In Section 3., we …

Content truncated to 500 words