Automating Feature Set Selection for Case-Based Learning of Linguistic Knowledge

Automating Feature Set Selection for Case-Based Learning of Linguistic Knowledge Claire Cardie Department of Computer Science Cornell University Ithaca, NY 14853-7501 E-mail: cardie@cs.cornell.edu Abstract This paper addresses the issue of "algorithm vs. representation" for case-based learning of lin- guistic knowledge. We first present empirical evidence that the success of case-based learning methods for natural language processing tasks de- pends to a large degree on the feature set used to describe the training instances. Next, we present a technique for automating feature set selection for case-based learning of linguistic knowledge. Given as input a baseline case representation, the method modifies the representation in response to a number of predefined linguistic biases by adding, deleting, and weighting features appro- priately. We apply the linguistic bias approach to feature set selection to the problem of relative pronoun disambiguation and show that the case- based learning algorithm improves as relevant bi- asses are incorporated into the underlying instance representation. Finally, we argue that the linguis- tic bias approach to feature set selection offers new possibilities for case-based learning of natural lan- guage: it simplifies the process of instance repre- sentation design and, in theory, obviates the need for separate instance representations for each lin- guistic knowledge acquisition task. More impor- tantly, the approach offers a mechanism for explic- itly combining the frequency information available from corpus-based techniques with linguistic bias information employed in traditional linguistic and knowledge-based approaches to natural language processing. Introduction Standard symbolic machine learning techniques have been successfully applied to a number of tasks in natural language processing (NLP). Ex- amples include the use of decision trees for syntac- tic analysis (Magerman, 1995), coreference (Aone and Bennett, 1995; McCarthy and Lehnert, 1995), and cue phrase identification (Litman, 1994); the use of inductive logic programming for learning semantic grammars and building prolog parsers 113 (Zelle and Mooney, 1994; Zelle and Mooney, 1993); the use of conceptual clustering algorithms for rel- ative pronoun resolution (Cardie, 1992a; Cardie 1992b), and the use of case-based learning tech- niques for lexical tagging tasks (Cardie, 1993a; Daelemans et al., submitted). In theory, both sta- tistical and machine learning techniques can sig- nificantly reduce the knowledge-engineering effort for building large-scale NLP systems: they offer an automatic means for acquiring robust heuris- tics for a host of lexical and structural disam- biguation tasks. It is well-known in the machine learning community, however, that the success of a learning algorithm depends critically on the representation used to describe the training and test instances (Almuallim and Dietterich, 1991, Langley and Sage, in press). Unfortunately, the task of designing an appropriate instance represen- tation -- also known as feature set selection -- can be extraordinarily difficult, time-consuming, and knowledge-intensive (Quinlan, 1983). This poses a problem for current statistical and machine learn- ing approaches to natural language understanding where a new instance representation is typically required for each linguistic task tackled. This paper addresses the role of the un- derlying instance representation for one class of symbolic machine learning algorithm as applied to …

Content truncated to 500 words