Learning Question Classifiers

Learning Question Classifiers � Xin Li Dan Roth Department of Computer Science University of Illinois at Urbana-Champaign fxli1,danr g@uiuc.edu Abstract In order to respond correctly to a free form factual ques- tion given a large collection of texts, one needs to un- derstand the question to a level that allows determining some of the constraints the question imposes on a pos- sible answer. These constraints may include a semantic classification of the sought after answer and may even suggest using different strategies when looking for and verifying a candidate answer. This paper presents a machine learning approach to question classification. We learn a hierarchical classi- fier that is guided by a layered semantic hierarchy of an- swer types, and eventually classifies questions into fine- grained classes. We show accurate results on a large col- lection of free-form questions used in TREC 10. 1 Introduction Open-domain question answering (Lehnert, 1986; Harabagiu et al., 2001; Light et al., 2001) and story comprehension (Hirschman et al., 1999) have be- come important directions in natural language pro- cessing. Question answering is a retrieval task more challenging than common search engine tasks be- cause its purpose is to find an accurate and concise answer to a question rather than a relevant docu- ment. The difficulty is more acute in tasks such as story comprehension in which the target text is less likely to overlap with the text in the questions. For this reason, advanced natural language techniques rather than simple key term extraction are needed. One of the important stages in this process is analyz- ing the question to a degree that allows determining the “type” of the sought after answer. In the TREC competition (Voorhees, 2000), participants are re- quested to build a system which, given a set of En- glish questions, can automatically extract answers (a short phrase) of no more than 50 bytes from a 5-gigabyte document library. Participants have re- � Research supported by NSF grants IIS-9801638 and ITR IIS- 0085836 and an ONR MURI Award. alized that locating an answer accurately hinges on first filtering out a wide range of candidates (Hovy et al., 2001; Ittycheriah et al., 2001) based on some categorization of answer types. This work develops a machine learning approach to question classification (QC) (Harabagiu et al., 2001; Hermjakob, 2001). Our goal is to categorize questions into different semantic classes that impose constraints on potential answers, so that they can be utilized in later stages of the question answering process. For example, when considering the ques- tion Q: What Canadian city has the largest popula- tion?, the hope is to classify this question as having answer type city, implying that only candidate an- swers that are cities need consideration. Based on the SNoW learning architecture, we de- velop a hierarchical classifier that is guided by a lay- ered semantic hierarchy of answer types and is able to classify questions into fine-grained classes. We suggest that it is useful to consider this classifica- tion task as a multi-label …

Content truncated to 500 words