Multi-Perspective Question Answering Using the OpQA Corpus

Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing (HLT/EMNLP), pages 923–930, Vancouver, October 2005. c⃝2005 Association for Computational Linguistics Multi-Perspective Question Answering Using the OpQA Corpus Veselin Stoyanov and Claire Cardie Department of Computer Science Cornell University Ithaca, NY 14850, USA {ves,cardie}@cs.cornell.edu Janyce Wiebe Department of Computer Science University of Pittsburgh Pittsburgh, PA 15260, USA wiebe@cs.pitt.edu Abstract We investigate techniques to support the answering of opinion-based questions. We first present the OpQA corpus of opin- ion questions and answers. Using the cor- pus, we compare and contrast the proper- ties of fact and opinion questions and an- swers. Based on the disparate characteris- tics of opinion vs. fact answers, we argue that traditional fact-based QA approaches may have difficulty in an MPQA setting without modification. As an initial step towards the development of MPQA sys- tems, we investigate the use of machine learning and rule-based subjectivity and opinion source filters and show that they can be used to guide MPQA systems. 1 Introduction Much progress has been made in recent years in automatic, open-domain question answering (e.g., Voorhees (2001), Voorhees (2002), Voorhees and Buckland (2003)). The bulk of the research in this area, however, addresses fact-based questions like: “When did McDonald’s open its first restaurant?” or “What is the Kyoto Protocol?”. To date, how- ever, relatively little research been done in the area of Multi-Perspective Question Answering (MPQA), which targets questions of the following sort: • How is Bush’s decision not to ratify the Kyoto Protocol looked upon by Japan and other US allies? • How do the Chinese regard the human rights record of the United States? In comparison to fact-based question answering (QA), researchers understand far less about the prop- erties of questions and answers in MPQA, and have yet to develop techniques to exploit knowledge of those properties. As a result, it is unclear whether approaches that have been successful in the domain of fact-based QA will work well for MPQA. We first present the OpQA corpus of opinion ques- tions and answers. Using the corpus, we compare and contrast the properties of fact and opinion ques- tions and answers. We find that text spans identi- fied as answers to opinion questions: (1) are approx- imately twice as long as those of fact questions, (2) are much more likely (37% vs. 9%) to represent par- tial answers rather than complete answers, (3) vary much more widely with respect to syntactic cate- gory – covering clauses, verb phrases, prepositional phrases, and noun phrases; in contrast, fact answers are overwhelming associated with noun phrases, and (4) are roughly half as likely to correspond to a sin- gle syntactic constituent type (16-38% vs. 31-53%). Based on the disparate characteristics of opinion vs. fact answers, we argue that traditional fact-based QA approaches may have difficulty in an MPQA setting without modification. As one such modifi- cation, we propose that MPQA systems should rely on natural language processing methods to identify information about opinions. In …

Content truncated to 500 words