Syntactic and Semantic Kernels for Short Text Pair Categorization

Proceedings of the 12th Conference of the European Chapter of the ACL, pages 576–584, Athens, Greece, 30 March – 3 April 2009. c⃝2009 Association for Computational Linguistics Syntactic and Semantic Kernels for Short Text Pair Categorization Alessandro Moschitti Department of Computer Science and Engineering University of Trento Via Sommarive 14 38100 POVO (TN) - Italy moschitti@disi.unitn.it Abstract Automatic detection of general relations between short texts is a complex task that cannot be carried out only relying on lan- guage models and bag-of-words. There- fore, learning methods to exploit syntax and semantics are required. In this pa- per, we present a new kernel for the repre- sentation of shallow semantic information along with a comprehensive study on ker- nel methods for the exploitation of syntac- tic/semantic structures for short text pair categorization. Our experiments with Sup- port Vector Machines on question/answer classification show that our kernels can be used to greatly improve system accuracy. 1 Introduction Previous work on Text Categorization (TC) has shown that advanced linguistic processing for doc- ument representation is often ineffective for this task, e.g. (Lewis, 1992; Furnkranz et al., 1998; Allan, 2000; Moschitti and Basili, 2004). In con- trast, work in question answering suggests that syntactic and semantic structures help in solving TC (Voorhees, 2004; Hickl et al., 2006). From these studies, it emerges that when the categoriza- tion task is linguistically complex, syntax and se- mantics may play a relevant role. In this perspec- tive, the study of the automatic detection of re- lationships between short texts is particularly in- teresting. Typical examples of such relations are given in (Giampiccolo et al., 2007) or those hold- ing between question and answer, e.g. (Hovy et al., 2002; Punyakanok et al., 2004; Lin and Katz, 2003), i.e. if a text fragment correctly responds to a question. In Question Answering, the latter problem is mostly tackled by using different heuristics and classifiers, which aim at extracting the best an- swers (Chen et al., 2006; Collins-Thompson et al., 2004). However, for definitional questions, a more effective approach would be to test if a cor- rect relationship between the answer and the query holds. This, in turns, depends on the structure of the two text fragments. Designing language mod- els to capture such relation is too complex since probabilistic models suffer from (i) computational complexity issues, e.g. for the processing of large bayesian networks, (ii) problems in effectively es- timating and smoothing probabilities and (iii) high sensitiveness to irrelevant features and processing errors. In contrast, discriminative models such as Support Vector Machines (SVMs) have theoreti- cally been shown to be robust to noise and irrele- vant features (Vapnik, 1995). Thus, partially cor- rect linguistic structures may still provide a rel- evant contribution since only the relevant infor- mation would be taken into account. Moreover, such a learning approach supports the use of kernel methods which allow for an efficient and effective representation of structured data. SVMs and Kernel Methods have recently been applied to natural language tasks with promising results, …

Content truncated to 500 words