A Graph-based Semi-Supervised Learning for Question-Answering

Proceedings of the 47th Annual Meeting of the ACL and the 4th IJCNLP of the AFNLP, pages 719–727, Suntec, Singapore, 2-7 August 2009. c⃝2009 ACL and AFNLP A Graph-based Semi-Supervised Learning for Question-Answering Asli Celikyilmaz EECS Department University of California at Berkeley Berkeley, CA, 94720 asli@berkeley.edu Marcus Thint Intelligent Systems Research Centre British Telecom (BT Americas) Jacksonville, FL 32256, USA marcus.2.thint@bt.com Zhiheng Huang EECS Department University of California at Berkeley Berkeley, CA, 94720 zhiheng@eecs.berkeley.edu Abstract We present a graph-based semi-supervised learning for the question-answering (QA) task for ranking candidate sentences. Us- ing textual entailment analysis, we obtain entailment scores between a natural lan- guage question posed by the user and the candidate sentences returned from search engine. The textual entailment between two sentences is assessed via features rep- resenting high-level attributes of the en- tailment problem such as sentence struc- ture matching, question-type named-entity matching based on a question-classifier, etc. We implement a semi-supervised learning (SSL) approach to demonstrate that utilization of more unlabeled data points can improve the answer-ranking task of QA. We create a graph for labeled and unlabeled data using match-scores of textual entailment features as similarity weights between data points. We apply a summarization method on the graph to make the computations feasible on large datasets. With a new representation of graph-based SSL on QA datasets using only a handful of features, and under lim- ited amounts of labeled data, we show im- provement in generalization performance over state-of-the-art QA models. 1 Introduction Open domain natural language question answer- ing (QA) is a process of automatically finding an- swers to questions searching collections of text files. There are intensive research in this area fostered by evaluation-based conferences, such as the Text REtrieval Conference (TREC) (Voorhees, 2004), etc. One of the focus of these research, as well as our work, is on factoid questions in En- glish, whereby the answer is a short string that in- dicates a fact, usually a named entity. A typical QA system has a pipeline structure starting from extraction of candidate sentences to ranking true answers. In order to improve QA systems’ performance many research focus on different structures such as question process- ing (Huang et al., 2008), information retrieval (Clarke et al., 2006), information extraction (Sag- gion and Gaizauskas, 2006), textual entailment (TE) (Harabagiu and Hickl, 2006) for ranking, an- swer extraction, etc. Our QA system has a sim- ilar pipeline structure and implements a new TE module for information extraction phase of the QA task. TE is a task of determining if the truth of a text entails the truth of another text (hypothesis). Harabagui and Hickl (2006) has shown that using TE for filtering or ranking answers can enhance the accuracy of current QA systems, where the an- swer of a question must be entailed by the text that supports the correctness of this answer. We derive information from pair of texts, i.e., question as hypothesis and candidate sentence as the text, potentially indicating containment of true answer, and …

Content truncated to 500 words