A Factoid Question Answering System Using Answer Pattern Matching

International Joint Conference on Natural Language Processing, pages 854–858, Nagoya, Japan, 14-18 October 2013. A Factoid Question Answering System Using Answer Pattern Matching Nagehan Pala Er Department of Computer Engineering, Bilkent University, Ankara, Turkey nagehan@cs.bilkent.edu.tr Ilyas Cicekli Department of Computer Engineering, Hacettepe University, Ankara, Turkey ilyas@cs.hacettepe.edu.tr Abstract In this paper, we describe a Turkish factoid QA system which uses surface level patterns called answer patterns in order to extract the answers from the documents that are retrieved from the web. The answer patterns are learned using five different answer pattern extraction methods with the help of the web. Our novel approach to extract named entity tagged an- swer patterns and our new confidence factor assignment approach have an important role in the successful performance of our QA system. We also describe a novel query expansion technique to improve the performance. The evaluation results show that the named entity tagging in answer patterns and the query ex- pansion leads to significant performance im- provement. The scores of our QA system are comparable with the results of the best factoid QA systems in the literature. 1 Introduction Question answering is the task of returning a particular piece of information in response to a natural language question. The aim of a question answering system is to present the required in- formation directly, instead of documents contain- ing potentially relevant information. Questions can be divided into five categories (Modolvan et. al., 2002; Schone et. al., 2005 ): factoid ques- tions, list questions, definition questions, com- plex questions, and speculative questions. A fac- toid question has exactly one correct answer which can be extracted from short text segments. The difficulty level of factoid questions is lower than the other categories. In this paper, we pre- sent a Turkish factoid question answering system which retrieves the documents that contain an- swers, and extracts the answers from these re- trieved documents with the help of a set of learned answer extraction patterns. List, defini- tion, complex, and speculative questions are out of the scope of this paper. At TREC-10 QA track (Voorhees, 2001), most of the question answering systems used Natural Language Processing (NLP) tools such as a natu- ral language parser and WordNet (Fellbaum, 1998). However, the best performing system at TREC-10 QA track used only an extensive list of surface patterns (Soubbotin and Soubbotin, 2001). Therefore we have decided to investigate their potential for Turkish factoid question an- swering. Our factoid question answering system learns answer patterns that are surface level pat- terns, and it uses them in the extraction of the answers of new questions. Our answer patterns are learned from the web using machine learning approaches. In addition to the creation of raw string answer patterns, we tried different meth- ods for answer pattern creation such as stemming and named entity tagging. Our novel answer pat- tern creation method using named entity tagging produces most successful results. One of the important issues in the factoid question answering is the answer ranking. The correct …

Content truncated to 500 words