UBBNBC WSD system description

UBBNBC WSD System Description Csomai ANDRÁS Department of Computer Science Babes-Bolyai University Cluj-Napoca, Romania csomaia@personal.ro Abstract The Naïve Bayes classification proves to be a good performing tool in word sense disambiguation, although it has not yet been applied to the Romanian language. The aim of this paper is to present our WSD system, based on the NBC algorithm, that performed quite well in Senseval 3. 1 Introduction According to the literature, the NBC algorithm is very efficient, in many cases it outperforms more sophisticated methods (Pedersen 1998). Therefore, this is the approach we used in our research. The word sense disambiguating process has three major steps, therefore, the application has three main components as follows: Stemming – removal of suffixes, and the filter- ing out of the irrelevant information from the corpora. A simple dictionary based ap- proach. Learning – the training of the classifier, based on the sense tagged corpora. A database containing the number of co-occurrences is built. Disambiguating –on the basis of the database, the correct sense of a word in a given con- text is estimated. In the followings the previously mentioned three steps are described in detail. 2 Stemming The preprocessing of the corpora is one of the most result-influential steps. The preprocessing consists of the removal of suffixes and the elimination of the irrelevant data. The removal of suffixes is per- formed trough a simple dictionary based method. For every wi word the possible wj candidates are selected from the dictionary containing the word stems. Then a similarity score is calculated be- tween the word to be stemmed and the candidates, as follows: li, lj is the length of word i, respectively j. scorei= j i li l 2+l if li ≤ lj and scorej=0, otherwise. The result is the candidate with the highest score if its score is above a certain threshold, otherwise the word is leaved untouched. In the preprocessing phase we also erase the pro- nouns and prepositions from the examined context. This exclusion was made upon a list of stop words. 3 Learning The training is conducted according to the NBC algorithm. First a database is built, with the follow- ing tables: words – contains all the words found in the cor- pora. Its role is to assign a sense id to every word. wordsenses – contains all the tagged words in the corpora linked with their possible senses. One entry for a given sense and word. Association for Computational Linguistics for the Semantic Analysis of Text, Barcelona, Spain, July 2004 SENSEVAL-3: Third International Workshop on the Evaluation of Systems nosenses - number of tagged contexts, with a given sense nocontexts - number of tagged contexts of a given word occurrences – number of co-occurrences of a given word with a given sense Figure1: The tables of the database The training of the system is nothing but filling up the tables of the database. fill NoSenses fill NoContexts fill Wordsenses scan corpora cakt=actual entry in corpora (a context) w=actual word …

Content truncated to 500 words