Using Machine Learning Techniques to Interpret WH-questions

Using Machine Learning Techniques to Interpret WH-questions Ingrid Zukerman School of Computer Science and Software Engineering Monash University Clayton, Victoria 3800, AUSTRALIA ingrid@csse.monash.edu.au Eric Horvitz Microsoft Research One Microsoft Way Redmond, WA 98052, USA horvitz@microsoft.com Abstract We describe a set of supervised ma- chine learning experiments centering on the construction of statistical mod- els of WH-questions. These models, which are built from shallow linguis- tic features of questions, are employed to predict target variables which repre- sent a user’s informational goals. We report on different aspects of the pre- dictive performance of our models, in- cluding the influence of various training and testing factors on predictive perfor- mance, and examine the relationships among the target variables. 1 Introduction The growth in popularity of the Internet highlights the importance of developing machinery for gen- erating responses to queries targeted at large un- structured corpora. At the same time, the access of World Wide Web resources by large numbers of users provides opportunities for collecting and leveraging vast amounts of data about user activ- ity. In this paper, we describe research on exploit- ing data collected from logs of users’ queries in order to build models that can be used to infer users’ informational goals from queries. We describe experiments which use supervised machine learning techniques to build statistical models of questions posed to the Web-based En- carta encyclopedia service. We focus on mod- els and analyses of complete questions phrased in English. These models predict a user’s infor- mational goals from shallow linguistic features of questions obtained from a natural language parser. We decompose these goals into (1) the type of information requested by the user (e.g., definition, value of an attribute, explanation for an event), (2) the topic, focal point and additional re- strictions posed by the question, and (3) the level of detail of the answer. The long-term aim of this project is to use predictions of these informational goals to enhance the performance of information- retrieval and question-answering systems. In this paper, we report on different aspects of the predic- tive performance of our statistical models, includ- ing the influence of various training and testing factors on predictive performance, and examine the relationships among the informational goals. In the next section, we review related research. In Section 3, we describe the variables being modeled. In Section 4, we discuss our predic- tive models. We then evaluate the predictions ob- tained from models built under different training and modeling conditions. Finally, we summarize the contribution of this work and discuss research directions. 2 Related Research Our research builds on earlier work on the use of probabilistic models to understand free-text queries in search applications (Heckerman and Horvitz, 1998; Horvitz et al., 1998), and on work conducted in the IR arena of question answering (QA) technologies. Heckerman and Horvitz (1998) and Horvitz et al. (1998) used hand-crafted models and super- vised learning to construct Bayesian models that predict users’ goals and needs for assistance in the context of consumer software …

Content truncated to 500 words