Framework and Results for the Spanish SENSEVAL

Framework and Results for the Spanish SENSEVAL German Rig au, Mariona Taule, Ana Fernandez and Julio Gonzalo g. rigau@lsi. upc. es, TALP Research Center, Universitat Politecnica de Catalunya rntaule@lingua. fiL ub. es, CLiC, Universitat de Barcelona ana. fernandez@uab. es, CLiC, Universitat Autonoma de Barcelona julio@lsi. uned. es, GPLN, Universidad Nacional de Educaci6n a Distancia Abstract In this paper we describe the structure, organisa- tion and results of the SENSEVAL exercise for Span- ish. We present several design decisions we taked for the exercise, we describe the creation of the gold- standard data and finally, we present the results of the evaluation. Twelve systems from five differ- ent universities were evaluated. Final scores ranged from 0.56 to 0.65. 1 Introduction In this paper we describe the structure, organisation and results of the Spanish exercise included within the framework of SENSEVAL-2. Although we closely follow the general architec- ture of the evaluation of SENSEVAL-2, the final setting of the Spanish exercise involved a number of choices detailed in section 2. In the following sec- tions we describe the data, the manual tagging pro- cess (including the inter-tagger agreement figures), the participant systems and the accuracy results (in- cluding some baselines for comparison purposes). 2 Design Decisions 2.1 Task Selection For Spanish SENSEVAL, the lexical-sample variant for the task was chosen. The main reasons for this decision are the following: • During the same tagging session, it is easier and quicker to concentrate only on one word at a time. That is, tagging multiple instances of the same word. • The all-words task requires access to a full dic- tionary. To our knowledge, there are no full Spanish dictionaries available (with low or no cost). Instead, the lexical-sample task required only as many dictionary entries as words in the sample task. 41 2.2 Word Selection The task for Spanish is a "lexical sample" for 39 words1 (17 nouns, 13 verbs, and 9 adjectives). See table 1 for the complete list of all words selected for the Spanish lexical sample task. The words can belong only to one of the syntactic categories. The fourteen words selected to be translation-equivalents to English has been: • Nouns: arte (=art), autoridad (= authority), canal ( = channel), circuito ( = circuit), and nat- uraleza ( = nature). • Verbs: conducir (=drive), tratar (=treat), and usar (=use). • Adjectives: ciego (=blind), local(= local), nat- ural (= natural), simple (= simple), verde (= green), and vital(= vital). 2.3 Corpus Selection The corpus was collected from two different sources: "El Peri6dico" 2 (a Spanish newspaper) and LexEsp3 (a balanced corpus of 5.5 million words). The length of corpus samples is the sentence. 2.4 Selection of Dictionary The lexicon provided was created specifically for the task and it consists of a definition for each sense linked to the Spanish version of EuroWordNet and, thus, to the English WordNet 1.5. The syntactic category and, sometimes, examples and synonyms are also provided. The connections to EuroWord- Net have been provided in order to …

Content truncated to 500 words