The Basque lexical-sample task

The Basque lexical-sample task Eneko Agirre, Itziar Aldabe, Mikel Lersundi, David Martinez, Eli Pociello, Larraitz Uria(*) IxA NLP group, Basque Country University 649 pk. 20.080 Donostia, Spain eneko@si.ehu.es Abstract In this paper we describe the Senseval 3 Basque lexical sample task. The task comprised 40 words (15 nouns, 15 verbs and 10 adjectives) selected from the Basque WordNet. 10 of the words were chosen in coordination with other lexical-sample tasks. The examples were taken from newspapers, an in-house balanced corpus and Internet texts. We additionally included a large set of untagged examples, and a lemmatised version of the data including lemma, PoS and case information. The method used to hand-tag the examples produced an inter-tagger agreement of 78.2% before arbitration. The eight competing systems attained results well above the most frequent baseline and the best system from Swarthmore College scored 70.4% recall. 1 Introduction This paper reviews the Basque lexical-sample task organized for Senseval 3. Each participant was provided with a relatively small set of labelled examples (2/3 of 75+15*senses+7*multiwords) and a comparatively large set of unlabelled examples (roughly ten times more when possible) for around 40 words. The larger number of unlabelled data was released with the purpose to enable the exploration of semi-supervised systems. The test set comprised 1/3 of the tagged examples. The sense inventory was taken from the Basque WordNet, which is linked to WordNet version 1.6 (Fellbaum, 1998). The examples came mainly from newspaper texts, although we also used a balanced in-house corpus and texts from Internet. The words selected for this task were coordinated with other lexical-sample tasks (such as Catalan, English, Italian, Romanian and Spanish) in order to share around 10 of the target words. The following steps were taken in order to carry out the task: (*) Authors listed in alphabetic order. 1. set the exercise a. choose sense inventory from a pre-existing resource b. choose target corpora c. choose target words d. lemmatize the corpus automatically e. select examples from the corpus 2. hand-tagging a. define the procedure b. revise the sense inventory c. tag d. analyze the inter-tagger agreement e. arbitrate This paper is organized as follows: The following section presents the setting of the exercise. Section 3 reviews the hand-tagging, and Section 4 the details of the final release. Section 5 shows the results of the participant systems. Section 6 discusses some main issues and finally, Section 7 draws the conclusions. 2 Setting of the exercise In this section we present the setting of the Basque lexical-sample exercise. 2.1 Basque As Basque is an agglutinative language, the dictionary entry takes each of the elements necessary to form the different functions. More specifically, the affixes corresponding to the determinant, number and declension case are taken in this order and independently of each other (deep morphological structure). For instance, ‘etxekoari emaiozu’ can be roughly translated as ‘[to the one in the house] [give it]’ where the underlined sequence of suffixes in Basque corresponds to ‘to the one in the’. 2.2 Sense …

Content truncated to 500 words