Using Syntactic Information for Improving Why-Question Answering

Proceedings of the 22nd International Conference on Computational Linguistics (Coling 2008), pages 953–960 Manchester, August 2008 Using Syntactic Information for Improving Why-Question Answering Suzan Verberne, Lou Boves, Nelleke Oostdijk and Peter-Arno Coppen Department of Linguistics Radboud University Nijmegen s.verberne@let.ru.nl Abstract In this paper, we extend an existing para- graph retrieval approach to why-question answering. The starting-point is a system that retrieves a relevant answer for 73% of the test questions. However, in 41% of these cases, the highest ranked relevant answer is not ranked in the top-10. We aim to improve the ranking by adding a re- ranking module. For re-ranking we con- sider 31 features pertaining to the syntactic structure of the question and the candidate answer. We find a significant improvement over the baseline for both success@10 and MRR@150. The most important features for re-ranking are the baseline score, the presence of cue words, the question’s main verb, and the relation between question fo- cus and document title. 1 Introduction Recently, some research has been directed at prob- lems involved in why-question answering (why- QA). About 5% of all questions asked to QA systems are why-questions (Hovy et al., 2002). They need a different approach from factoid ques- tions, since their answers cannot be stated in a sin- gle phrase. Instead, a passage retrieval approach seems more suitable. In (Verberne et al., 2008), we proposed an approach to why-QA that is based on paragraph retrieval. We reported mediocre per- formance and suggested that adding linguistic in- formation may improve ranking power. c⃝ Suzan Verberne, 2008. Licensed under the Creative Commons Attribution-Noncommercial-Share Alike 3.0 Un- ported license (http://creativecommons.org/licenses/by-nc- sa/3.0/). Some rights reserved. In the present paper, we implement a simi- lar paragraph retrieval approach and extend it by adding a re-ranking module based on structural lin- guistic information. Our aim is to find out whether syntactic knowledge is relevant for discovering re- lations between question and answer, and if so, which type of information is the most beneficial. In the following sections, we first discuss related work (section 2). In sections 3 and 4, we introduce the data that we used for development purposes and the baseline retrieval and ranking method that we implemented. In section 5, we present our re- ranking method and the results obtained, followed by a discussion in section 6, and directions for fur- ther research in section 7. 2 Related work A substantial amount of work has been done in improving QA by adding syntactic information (Tiedemann, 2005; Quarteroni et al., 2007; Hi- gashinaka and Isozaki, 2008). All these studies show that syntactic information gives a small but significant improvement on top of the traditional bag-of-words (BOW) approaches. The work of (Higashinaka and Isozaki, 2008) focuses on the problem of ranking candidate an- swer paragraphs for Japanese why-questions. They find a success@10 score of 70.3% with an MRR of 0.328. They conclude that their system for Japanese is the best-performing fully implemented why-QA system. In (Tiedemann, 2005), passage retrieval for Dutch factoid QA is …

Content truncated to 500 words