Exploiting Syntactic and Shallow Semantic Kernels for Question Answer Classification

Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pages 776–783, Prague, Czech Republic, June 2007. c⃝2007 Association for Computational Linguistics Exploiting Syntactic and Shallow Semantic Kernels for Question/Answer Classification Alessandro Moschitti University of Trento 38050 Povo di Trento Italy moschitti@dit.unitn.it Silvia Quarteroni The University of York York YO10 5DD United Kingdom silvia@cs.york.ac.uk Roberto Basili “Tor Vergata” University Via del Politecnico 1 00133 Rome, Italy basili@info.uniroma2.it Suresh Manandhar The University of York York YO10 5DD United Kingdom suresh@cs.york.ac.uk Abstract We study the impact of syntactic and shallow semantic information in automatic classifi- cation of questions and answers and answer re-ranking. We define (a) new tree struc- tures based on shallow semantics encoded in Predicate Argument Structures (PASs) and (b) new kernel functions to exploit the representational power of such structures with Support Vector Machines. Our ex- periments suggest that syntactic information helps tasks such as question/answer classifi- cation and that shallow semantics gives re- markable contribution when a reliable set of PASs can be extracted, e.g. from answers. 1 Introduction Question answering (QA) is as a form of informa- tion retrieval where one or more answers are re- turned to a question in natural language in the form of sentences or phrases. The typical QA system ar- chitecture consists of three phases: question pro- cessing, document retrieval and answer extraction (Kwok et al., 2001). Question processing is often centered on question classification, which selects one of k expected an- swer classes. Most accurate models apply super- vised machine learning techniques, e.g. SNoW (Li and Roth, 2005), where questions are encoded us- ing various lexical, syntactic and semantic features. The retrieval and answer extraction phases consist in retrieving relevant documents (Collins-Thompson et al., 2004) and selecting candidate answer passages from them. A further answer re-ranking phase is op- tionally applied. Here, too, the syntactic structure of a sentence appears to provide more useful infor- mation than a bag of words (Chen et al., 2006), al- though the correct way to exploit it is still an open problem. An effective way to integrate syntactic structures in machine learning algorithms is the use of tree ker- nel (TK) functions (Collins and Duffy, 2002), which have been successfully applied to question classifi- cation (Zhang and Lee, 2003; Moschitti, 2006) and other tasks, e.g. relation extraction (Zelenko et al., 2003; Moschitti, 2006). In more complex tasks such as computing the relatedness between questions and answers in answer re-ranking, to our knowledge no study uses kernel functions to encode syntactic in- formation. Moreover, the study of shallow semantic information such as predicate argument structures annotated in the PropBank (PB) project (Kingsbury and Palmer, 2002) (www.cis.upenn.edu/∼ace) is a promising research direction. We argue that seman- tic structures can be used to characterize the relation between a question and a candidate answer. In this paper, we extensively study new structural representations, encoding parse trees, bag-of-words, POS tags and predicate argument structures (PASs) for question classification and answer re-ranking. We define new tree representations for both …

Content truncated to 500 words