Exceptionality and Natural Language Learning

Exceptionality and Natural Language Learning Mihai Rotaru Diane J. Litman Computer Science Department University of Pittsburgh Pittsburgh, PA 15260 mrotaru, litman @cs.pitt.edu Abstract Previous work has argued that memory-based learning is better than abstraction-based learn- ing for a set of language learning tasks. In this paper, we first attempt to generalize these re- sults to a new set of language learning tasks from the area of spoken dialog systems and to a different abstraction-based learner. We then examine the utility of various exceptionality measures for predicting where one learner is better than the other. Our results show that generalization of previous results to our tasks is not so obvious and some of the exceptional- ity measures may be used to characterize the performance of our learners. 1 Introduction 2 Our paper is a follow-up of the study done by Daele- mans et al. (1999) in which the authors show that keep- ing exceptional training instances is useful for increasing generalization accuracy when natural lan- guage learning tasks are involved. The tasks used in their experiments are: grapheme-phoneme conversion, part of speech tagging, prepositional phrase attachment and base noun phrase chunking. Their study provides empirical evidence that editing exceptional instances leads to a decrease in memory-based learner perform- ance. Next, the memory-based learner is compared on the same tasks with a decision-tree learner and their results favor the memory-based learner. Moreover, the authors provide evidence that the performance of their memory-based learner is linked to its property of hold- ing all instances (including exceptional ones) and gen- eral properties of language learning tasks (difficultness in discriminating between noise and valid exceptions and sub-regularities for those tasks). We continue on the same track by investigating if their results hold on a different set of tasks. Our tasks come from the area of spoken dialog systems and have smaller datasets and more features (with many of the features being numeric, in contrast with the previous study that had none). We observe in our experiments with these tasks a much smaller exceptionality measure range compared with the previous study. Our results indicate that the previous results do not generalize to all our tasks. An additional goal of our research is to investigate a new topic by looking into whether exceptionality meas- ures can be used to characterize the performance of our learners: a memory-based learner (IB1-IG) and a rule- based learner (Ripper). Our results indicate that for some of the exceptionality measures we will examine, IB1-IG is better for predicting typical instances while Ripper is better for predicting exceptional instances. We will use the following conventions throughout the paper. The term “exceptional” will be used to label instances that do not follow the rules that characterize the class they are part of (in language learning terms, they are “bad” examples of their class rules). We will use “typical” as the antonym of this term; it will label instances that are good examples of their class rules. The fact that an instance is typical should …

Content truncated to 500 words