Through Low-Cost Annotation to Reliable Parsing Evaluation

Through Low-Cost Annotation to Reliable Parsing Evaluation Marek Grác, Miloš Jakubíˇcek, Vojtˇech Kováˇr {xgrac,xjakub,xkovar3}@fi.muni.cz Natural Language Processing Centre, Faculty of Informatics Masaryk University, Brno, Czech Republic Abstract. In this paper, we present an application-driven low-cost concept of building a multi-purpose language resource for Czech which is based on currently available results of previous work by various research teams active in the area of natural language process- ing. We particularly focus on the first phase which consists in extracting noun phrases from a morphologically annotated corpus and providing a simple and easy-to-use application for verifying them. For the extraction task, three Czech parsers have been accommodated and evaluated. Finally we discuss the currently achieved results in the context of ongoing work and show that they lead to consistent and reliable results. Keywords: noun phrases, parsing, parser evaluation, annotation, inter-annotator agreement 1 Introduction Language resources for natural language processing are very important for development as well as improvement of existing natural language processing (NLP) tools. Situation for different European languages ranges from very good to almost non-existent. In the worst case there are almost no resources and we face the problem of creating them in a cheap and fast way while providing high quality results. An important aspect of building language resources is that one should always keep in mind the usability of the resource for particular applications. This approach, known as application-driven development, can already in early development stages prevent major design flaws which might not be automatically recoverable later on and could limit the usefulness of the resulting work. In this approach, we present a language resource – an annotated corpus – as an intermediate step for improving many NLP applications (some of them even before full completion of the corpus). This kind of development helps us to create data which does not necessarily attempt to cover all nuances of natural language but strictly concentrates on particular tasks we want to solve. Czech language is one of the most described European languages and there are several NLP teams that compete to build optimal language resources and provide cutting-edge applications on top of them. Our main goal was to create a new language resource based partially on existing data. We chose to use morphologically annotated corpus DESAM (Pala et al., 1997), that was manually checked by annotators. On top of this corpus we want to build a multi-layer annotation. 2 The Annotation Project 2.1 General Overview The project aims to provide a multi-purpose annotated corpus for enhancing several NLP applica- tions. In the first phase we want to perform computer aided syntactic annotation of noun phrases (that will be described further on). As the next step those noun phrases will be matched to the complex valency frames available in the Czech verb valency lexicon VerbaLex (Horák and Pala, 2007). Among other benefits, valency frames in this lexicon contain semantic information in the form of so called semantic roles as present in the Czech WordNet (Pala and Smrž, 2004). This can PACLIC …

Content truncated to 500 words