Probabilistic Tree-Edit Models with Structured Latent Variables for Textual Entailment and Question Answering

Proceedings of the 23rd International Conference on Computational Linguistics (Coling 2010), pages 1164–1172, Beijing, August 2010 Probabilistic Tree-Edit Models with Structured Latent Variables for Textual Entailment and Question Answering Mengqiu Wang Computer Science Department Stanford University mengqiu@cs.stanford.edu Christopher D. Manning Computer Science Department Stanford University manning@cs.stanford.edu Abstract A range of Natural Language Process- ing tasks involve making judgments about the semantic relatedness of a pair of sen- tences, such as Recognizing Textual En- tailment (RTE) and answer selection for Question Answering (QA). A key chal- lenge that these tasks face in common is the lack of explicit alignment annota- tion between a sentence pair. We capture the alignment by using a novel probabilis- tic model that models tree-edit operations on dependency parse trees. Unlike previ- ous tree-edit models which require a sep- arate alignment-finding phase and resort to ad-hoc distance metrics, our method treats alignments as structured latent vari- ables, and offers a principled framework for incorporating complex linguistic fea- tures. We demonstrate the robustness of our model by conducting experiments for RTE and QA, and show that our model performs competitively on both tasks with the same set of general features. 1 Introduction Many complex Natural Language Processing (NLP) applications can be broken down to a sub- task of evaluating the semantic relationship of pairs of sentences (e.g., in Question Answering, answer selection involve comparing each answer candidate against the question). This means that research aiming at analyzing pairs of semanti- cally related natural language sentences is promis- ing because of its reusability: it is not tied to a particular internal representation of meanings, but it nevertheless serves as a first step towards full meaning understanding, which is applicable to a number of applications. At the same time, this paradigm clearly defines the input and output space, facilitating system comparison and stan- dard evaluation. Tasks of this paradigm have drawn much of the focus in recent NLP research, including Recognizing Textual Entailment (RTE), answer selection for Question Answering (QA), Paraphrase Identification (PI), Machine Transla- tion Evaluation (MTE), and many more. In each of these tasks, inputs to the systems are pairs of sentences that may or may not convey the desired semantic property (e.g., in RTE, whether the hypothesis sentence can be entailed from the premise sentence; in QA, whether the answer can- didate sentence correctly answers the question), and the output of the system is a binary classifi- cation decision (or a regression score,as in MTE). Earlier studies in these domains have concluded that simple word overlap measures (e.g., bag of words, n-grams) have a surprising degree of util- ity (Papineni et al., 2002; Jijkoun and de Ri- jke, 2005b), but are nevertheless not sufficient for these tasks (Jijkoun and de Rijke, 2005a). A com- mon problem identified in these earlier systems is the lack of understanding of the semantic relation between words and phrases. Later systems that include more linguistic features extracted from re- sources such as WordNet have enjoyed more suc- cess (MacCartney et al., 2006). …

Content truncated to 500 words