Empirical Evaluations of Animacy Annotation

Proceedings of the 12th Conference of the European Chapter of the ACL, pages 630–638, Athens, Greece, 30 March – 3 April 2009. c⃝2009 Association for Computational Linguistics Empirical evaluations of animacy annotation Lilja Øvrelid Department of Linguistics University of Potsdam Germany lilja@ling.uni-potsdam.de Abstract This article presents empirical evaluations of aspects of annotation for the linguis- tic property of animacy in Swedish, rang- ing from manual human annotation, auto- matic classification and, finally, an exter- nal evaluation in the task of syntactic pars- ing. We show that a treatment of animacy as a lexical semantic property of noun types enables generalization over distri- butional properties of these nouns which proves beneficial in automatic classifica- tion and furthermore gives significant im- provements in terms of parsing accuracy for Swedish, compared to a state-of-the- art baseline parser with gold standard ani- macy information. 1 Introduction The property of animacy influences linguistic phe- nomena in a range of different languages, such as case marking (Aissen, 2003) and argument re- alization (Bresnan et al., 2005; de Swart et al., 2008), and has been shown to constitute an im- portant factor in the production and comprehen- sion of syntactic structure (Branigan et al., 2008; Weckerly and Kutas, 1999).1 In computational linguistic work, animacy has been shown to pro- vide important information in anaphora resolution (Or˘asan and Evans, 2007), argument disambigua- tion (Dell’Orletta et al., 2005) and syntactic pars- ing in general (Øvrelid and Nivre, 2007). The dimension of animacy roughly distin- guishes between entities which are alive and en- tities which are not, however, other distinctions 1Parts of the research reported in this paper has been sup- ported by the Deutsche Forschungsgemeinschaft (DFG, Son- derforschungsbereich 632, project D4). are also relevant and the animacy dimension is of- ten viewed as a continuum ranging from humans to inanimate objects. Following Silverstein (1976) several animacy hierarchies have been proposed in typological studies, focusing on the linguistic cat- egory of animacy, i.e., the distinctions which are relevant for linguistic phenomena. An example of an animacy hierarchy, taken from (Aissen, 2003), is provided in (1): (1) Human > Animate > Inanimate Clearly, non-human animates, like animals, are not less animate than humans in a biological sense, however, humans and animals show differing lin- guistic behaviour. Empirical studies of animacy require human an- notation efforts, and, in particular, a well-defined annotation task. However, annotation studies of animacy differ distinctly in their treatment of ani- macy as a type or token-level phenomenon, as well as in terms of granularity of categories. The use of the annotated data as a computational resource furthermore poses requirements on the annotation which do not necessarily agree with more theo- retical considerations. Methods for the induction of animacy information for use in practical appli- cations require the resolution of issues of level of representation, as well as granularity. This article addresses these issues through em- pirical and experimental evaluation. We present an in-depth study of a manually annotated data set which indicates that animacy may be treated …

Content truncated to 500 words