A New Scheme for Annotating Semantic Relations between Named Entities in Corpora

Proceedings of Recent Advances in Natural Language Processing, pages 275–281, Hissar, Bulgaria, 12-14 September 2011. A New Scheme for Annotating Semantic Relations between Named Entities in Corpora Mani EZZAT(1)(2) (1)INaLCO-CRIM 49bis av. Belle Gabrielle 75012 Paris mani.ezzat@gmail.com (2) ARISEM 1-5 rue Carnot 91883 Massy Thierry POIBEAU(3) (3) LaTTiCe-CNRS and ENS 1, rue Maurice Arnoux 92120 Montrouge, France thierry.poibeau@ens.fr Abstract Although several studies have developed models and type hierarchies for named en- tity annotation, no such resource is avail- able for semantic relation annotation, de- spite its utility for various applications (e.g. question answering, information ex- traction). In this paper, we show that there are two issues in semantic relation de- scription, one concerning knowledge engi- neering (what to annotate?) and the other concerning language engineering (how to deal with modality and modifiers?). We propose a new annotation scheme, mak- ing it possible to have both a precise and tractable annotation. A practical exper- iment shows that annotators using our scheme were able to quickly annotate a large number of sentences with very high inter-annotator agreement. 1 Introduction A large number of natural language applications (e.g. information extraction, question answering, automatic summarization) require a precise analy- sis of the linguistic content of the text. Since the Message Understanding Conferences in the 1990s, there is a general agreement on the different steps required to perform this analysis: i) relevant ele- ments (mostly named entities) are first recognized and tagged, then ii) relations between these ele- ments are extracted. This generic schema does not preclude the existence of other steps in the anal- ysis (e.g. anaphora resolution, discourse structure recognition), but the recognition of basic elements and relations between them is nevertheless a shared basis among a large number of systems (Jurafsky and Martin, 2009). This of course explains why there has been an increasing amount of research both on named en- tity recognition and on relation analysis in the last 20 years (MUC6, 1995; Appelt and Martin, 1999). However, the maturity of these two tasks differs to a large extent. As for named entity recognition, a large number of tools, data and gold standard are available for very different languages. The success rate is often above .9 or even .95 F-measure for ma- jor categories (person’s names, location’s names) in newspapers (Collins and Singer, 1999). Entity types are to a certain extent normalized and for- malized in large hierarchies (see for example the hierarchy proposed by Sekine which is now a de facto standard (Sekine et al., 2002)). In comparison, it is interesting to observe that only a few annotated corpora and no real gold standard exist for semantic relations1. A first ex- planation is that relation analysis largely depends on the task and on the kind of corpora being an- alyzed. However, we do not think that this is enough to explain the current situation: for ex- ample question-answering systems are supposed to address any kinds of questions and thus require a generic approach for relation analysis. It is of course …

Content truncated to 500 words