Semantic Classification with Distributional Kernels

Proceedings of the 22nd International Conference on Computational Linguistics (Coling 2008), pages 649–656 Manchester, August 2008 Semantic Classification with Distributional Kernels Diarmuid ´O S´eaghdha Computer Laboratory University of Cambridge 15 JJ Thomson Avenue Cambridge CB3 0FD United Kingdom do242@cl.cam.ac.uk Ann Copestake Computer Laboratory University of Cambridge 15 JJ Thomson Avenue Cambridge CB3 0FD United Kingdom aac10@cl.cam.ac.uk Abstract Distributional measures of lexical similar- ity and kernel methods for classification are well-known tools in Natural Language Processing. We bring these two meth- ods together by introducing distributional kernels that compare co-occurrence prob- ability distributions. We demonstrate the effectiveness of these kernels by present- ing state-of-the-art results on datasets for three semantic classification: compound noun interpretation, identification of se- mantic relations between nominals and se- mantic classification of verbs. Finally, we consider explanations for the impressive performance of distributional kernels and sketch some promising generalisations. 1 Introduction This paper draws a connection between two well- known topics in statistical Natural Language Pro- cessing: distributional measures of lexical simi- larity and kernel methods for classification. Dis- tributional similarity measures quantify the sim- ilarity between pairs of words through their ob- served co-occurrences with other words in corpus data. The kernel functions used in support vec- tor machine classifiers also allow an interpretation as similarity measures; however, not all similar- ity measures can be used as kernels. In particu- lar, kernel functions must satisfy the mathemati- cal property of positive semi-definiteness. In Sec- tion 2 we consider kernel functions suitable for comparing co-occurrence probability distributions and show that these kernels are closely related to measures known from the distributional similarity literature. We apply these distributional kernels © 2008. Licensed under the Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported li- cense (http://creativecommons.org/licenses/ by-nc-sa/3.0/). Some rights reserved. to three semantic classification tasks: compound noun interpretation, identification of semantic re- lations between nominals and semantic classifica- tion of verbs. In all cases, the distributional ker- nels outperform the linear and Gaussian kernels standardly used for SVM classification and fur- thermore achieve state-of-the-art results. In Sec- tion 4 we provide a concrete explanation for the superior performance of distributional kernels, and in Section 5 we outline some promising directions for future research. 2 Theory 2.1 Distributional Similarity Measures Distributional approaches to lexical similarity as- sume that words appearing in similar contexts are likely to have similar or related meanings. To measure distributional similarity, we use a repre- sentation of words based on observation of their relations with other words. Specifically, a target word w is represented in terms of a set C of ad- missible co-occurrence types c = (r, w′), where the word w′ belongs to a co-occurrence vocab- ulary Vc and r is a relation that holds between w and w′. Co-occurrence relations may be syn- tactic (e.g., verb-argument, conjunct-conjunct) or may simply be one of proximity in text. Counts f(w, c) of a target word w’s co-occurrences can be estimated from language corpora, and these counts can be weighted in a variety of ways to re- flect …

Content truncated to 500 words