Selecting Sentences for Answering Complex Questions

Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing, pages 304–313, Honolulu, October 2008. c⃝2008 Association for Computational Linguistics Selecting Sentences for Answering Complex Questions Yllias Chali University of Lethbridge 4401 University Drive Lethbridge, Alberta, Canada, T1K 3M4 chali@cs.uleth.ca Shafiq R. Joty University of British Columbia 2366 Main Mall Vancouver, B.C. Canada V6T 1Z4 rjoty@cs.ubc.ca Abstract Complex questions that require inferencing and synthesizing information from multiple documents can be seen as a kind of topic- oriented, informative multi-document summa- rization. In this paper, we have experimented with one empirical and two unsupervised statistical machine learning techniques: k- means and Expectation Maximization (EM), for computing relative importance of the sen- tences. However, the performance of these ap- proaches depends entirely on the feature set used and the weighting of these features. We extracted different kinds of features (i.e. lex- ical, lexical semantic, cosine similarity, ba- sic element, tree kernel based syntactic and shallow-semantic) for each of the document sentences in order to measure its importance and relevancy to the user query. We used a local search technique to learn the weights of the features. For all our methods of generating summaries, we have shown the effects of syn- tactic and shallow-semantic features over the bag of words (BOW) features. 1 Introduction After having made substantial headway in factoid and list questions, researchers have turned their at- tention to more complex information needs that can- not be answered by simply extracting named enti- ties (persons, organizations, locations, dates, etc.) from documents. For example, the question: “De- scribe the after-effects of cyclone Sidr-Nov 2007 in Bangladesh” requires inferencing and synthesizing information from multiple documents. This infor- mation synthesis in NLP can be seen as a kind of topic-oriented, informative multi-document summa- rization, where the goal is to produce a single text as a compressed version of a set of documents with a minimum loss of relevant information. In this paper, we experimented with one em- pirical and two well-known unsupervised statisti- cal machine learning techniques: k-means and EM and evaluated their performance in generating topic- oriented summaries. However, the performance of these approaches depends entirely on the feature set used and the weighting of these features. We ex- tracted different kinds of features (i.e. lexical, lexi- cal semantic, cosine similarity, basic element, tree kernel based syntactic and shallow-semantic) for each of the document sentences in order to measure its importance and relevancy to the user query. We have used a gradient descent local search technique to learn the weights of the features. Traditionally, information extraction techniques are based on the BOW approach augmented by language modeling. But when the task requires the use of more com- plex semantics, the approaches based on only BOW are often inadequate to perform fine-level textual analysis. Some improvements on BOW are given by the use of dependency trees and syntactic parse trees (Hirao et al., 2004), (Punyakanok et al., 2004), (Zhang and Lee, 2003), but these, too are not ade- quate when dealing with …

Content truncated to 500 words