Complementarity of lexical and simple syntactic features: The SyntaLex approach to Senseval-3

Complementarity of Lexical and Simple Syntactic Features: The SyntaLex Approach to SENSEVAL-3 Saif Mohammad University of Toronto Toronto, ON M5S1A1 Canada smm@cs.toronto.edu http://www.cs.toronto.edu/˜smm Ted Pedersen University of Minnesota Duluth, MN 55812 USA tpederse@d.umn.edu http://www.d.umn.edu/˜tpederse Abstract This paper describes the SyntaLex entries in the English Lexical Sample Task of SENSEVAL-3. There are four entries in all, where each of the dif- ferent entries corresponds to use of word bigrams or Part of Speech tags as features. The systems rely on bagged decision trees, and focus on using pairs of lexical and syntactic features individually and in combination. They are descendants of the Duluth systems that participated in SENSEVAL-2. 1 Introduction The SyntaLex systems are supervised learners that identify the intended sense of a word (target word) given its context. They are derived from the Duluth systems that participated in SENSEVAL-2, and which are more fully described in (Pedersen, 2001b). The context of a word is a rich source of dis- crete features which lend themselves nicely to de- cision tree learning. Prior research (e.g., (McRoy, 1992), (Ng and Lee, 1996), (Stevenson and Wilks, 2001), (Yarowsky and Florian, 2002)) suggests that use of both syntactic and lexical features will im- prove disambiguation accuracies. There has also been considerable work on word sense disambigua- tion using various supervised learning algorithms. However, both (Pedersen, 2001a) and (Lee and Ng, 2002) show that different learning algorithms pro- duce similar results and that the use of appropriate features may dramatically improve results. Thus, our focus is not on the learning algorithm but on the features used and their dynamics. Our systems use bigrams and Part of Speech fea- tures individually, in a simple ensemble and as part of single classifier using both kinds of features. We also show that state of the art results (72.1%, coarse grained accuracy) can be achieved using just these simple sets of features. 2 Feature Space Simple lexical and syntactic features are used to rep- resent the context. The lexical features used are word bigrams. The Part of Speech (PoS) of the tar- get word and its neighbors make up the the syntactic features. Bigrams are readily captured from the text while Part of Speech taggers are widely available for a variety of languages. 2.1 Bigrams A bigram is a pair of words that occur close to each other in text and in a particular order. Consider: ������ � �� � ��� �� �� � �� � �� ! (1) It has the following bigrams: the interest, interest rate, rate is, is lower, lower in, in state and state banks. Note that the bigram interest rate suggests that bank has been used in the financial institution sense and not the river bank sense. All features are binary valued. Thus, the bi- gram feature interest rate has value 1 if it occurs in the context of the target word, and 0 if it does not. The learning algorithm considers only those bigrams that occur at least twice in the training data and …

Content truncated to 500 words