MELB-MKB: Lexical Substitution system based on Relatives in Context

Proceedings of the 4th International Workshop on Semantic Evaluations (SemEval-2007), pages 237–240, Prague, June 2007. c⃝2007 Association for Computational Linguistics MELB-MKB: Lexical Substitution System based on Relatives in Context David Martinez, Su Nam Kim and Timothy Baldwin LT Group, CSSE University of Melbourne Victoria 3010 Australia {davidm,snkim,tim}@csse.unimelb.edu.au Abstract In this paper we describe the MELB-MKB system, as entered in the SemEval-2007 lex- ical substitution task. The core of our sys- tem was the “Relatives in Context” unsuper- vised approach, which ranked the candidate substitutes by web-lookup of the word se- quences built combining the target context and each substitute. Our system ranked third in the final evaluation, performing close to the top-ranked system. 1 Introduction This paper describes the system we developed for the SemEval lexical substitution task, a new task in SemEval-2007. Although we tested different con- figurations on the trial data, our basic system relied on WordNet relatives (Fellbaum, 1998) and Google queries in order to identify the most plausible sub- stitutes in the context. The main goal when building our system was to study the following factors: (i) substitution candi- date set, (ii) settings of the relative-based algorithm, and (iii) syntactic filtering. We analysed these fac- tors over the trial data provided by the organisation, and used the BEST metric to tune our system. This metric accepts multiple answers, and averages the score across the answers. We did not experiment with the OOT (top 10 answers) and MULTIWORD metrics. In the remainder of this paper we briefly intro- duce the basic Relatives in Context algorithm in Sec- tion 2. Next we describe our experiments on the trial data in Section 3. Our final system and its results are described in Section 4. Finally, our conclusions are outlined in Section 5. 2 Algorithm Our basic algorithm is an unsupervised method pre- sented in Martinez et al. (2006). This technique makes use of the WordNet relatives of the target word for disambiguation, by way of the following steps: (i) obtain a set of close relatives from Word- Net for each sense of the target word; (ii) for each test instance define all possible word sequences that include the target word; (iii) for each word sequence, substitute the target word with each relative, and then query Google; (iv) rank queries according to the following factors: length of the query, distance of the relative to the target word, and number of hits; and (v) select the relative from the highest ranked query.1 For the querying step, first we tokenise each tar- get sentence, and then we apply sliding windows of different sizes (up to 6 tokens) that include the tar- get word. For each window and each relative in the pool, we substitute the target word for the relative, and query Google. The algorithm stops augment- ing the window for the relative when one of its sub- strings returns zero hits. The length of the query is measured as the number of words, and the distance of the relative to the …

Content truncated to 500 words