Learning the Relative Usefulness of Questions in Community QA

Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pages 97–107, MIT, Massachusetts, USA, 9-11 October 2010. c⃝2010 Association for Computational Linguistics Learning the Relative Usefulness of Questions in Community QA Razvan Bunescu School of EECS Ohio University Athens, OH 43201, USA bunescu@ohio.edu Yunfeng Huang School of EECS Ohio University Athens, OH 43201, USA yh324906@ohio.edu Abstract We present a machine learning approach for the task of ranking previously answered ques- tions in a question repository with respect to their relevance to a new, unanswered refer- ence question. The ranking model is trained on a collection of question groups manually annotated with a partial order relation reflect- ing the relative utility of questions inside each group. Based on a set of meaning and struc- ture aware features, the new ranking model is able to substantially outperform more straight- forward, unsupervised similarity measures. 1 Introduction Open domain Question Answering (QA) is one of the most complex and challenging tasks in natural language processing. In general, a question answer- ing system may need to integrate knowledge coming from a wide variety of linguistic processing tasks such as syntactic parsing, semantic role labeling, named entity recognition, and anaphora resolution (Prager, 2006). State of the art implementations of these linguistic analysis tasks are still limited in their performance, with errors that compound and prop- agate into the final performance of the QA system (Moldovan et al., 2002). Consequently, the perfor- mance of open domain QA systems has yet to ar- rive at a level at which it would become a feasible alternative to the current paradigms for information access based on keyword searches. Recently, community-driven QA sites such as Ya- hoo! Answers and WikiAnswers 1 have established 1answers.yahoo.com, wiki.answers.com a new approach to question answering that shifts the inherent complexity of open domain QA from the computer system to volunteer contributors. The computer is no longer required to perform a deep linguistic analysis of questions and generate corre- sponding answers, and instead acts as a mediator between users submitting questions and volunteers providing the answers. An important objective in community QA is to minimize the time elapsed between the submission of questions by users and the subsequent posting of answers by volunteer contributors. One useful strategy for minimizing the response latency is to search the QA repository for similar questions that have already been answered, and provide the cor- responding ranked list of answers, if such a ques- tion is found. The success of this approach de- pends on the definition and implementation of the question-to-question similarity function. In the sim- plest solution, the system searches for previously answered questions based on exact string match- ing with the reference question. Alternatively, sites such as WikiAnswers allow the users to mark ques- tions they think are rephrasings (“alternate word- ings”, or paraphrases) of existing questions. These question clusters are then taken into account when performing exact string matching, therefore increas- ing the likelihood of finding previously answered questions that are semantically equivalent to …

Content truncated to 500 words