Do Not Forget: Full Memory in Memory-Based Learning of Word Pronunciation

/ / / I / / / Do Not Forget: Full Memory in Memory-Based Learning of Word Pronunciation * Antal van den Bosch and Walter Daelemans Tilburg University, ILK P.O. Box 90153, NL-5000 LE Tilburg The Netherlands {ant alb, ealt er}@kub, nl Abstract Memory-based learning, keeping full memory ofleaxning material, appeaxs a viable approach to learning N-~ tasks, and is often superior in genera~sation accuracy to eager learning approaches that abstract from learning mate- riaL Here we investigate three pa~'tial memory- based learning approaches which remove from memory specific task instance types estimated to be exceptional. The three approaches each implement one heuristic function for estimat- ing exceptiona]ity of instance types: (i) typi- catty, (ii) class prediction strength, and (fii) friencfly-neighbourhood size. Experiments are performed with the memory-based learning al- gorithm IBI-IG trained on English word pro- nunciatlon. We find that removing instance types with low prediction strength (il) is the only tested method which does not seriously harm generallsation accuracy. We conclude that keeping full memory of types rather than tokens, and excluding minority ambiguities ap- pear to be the only performance-preserving op- timi~tions of memory-based leaxning. 1 Introduction Memory-based learning of classification tasks is a branch of supervised machine learning in which the learning phase consists simply of storing all en- countered instances from a training set in mem- ory (Aha, 1997). Memory-based learning algorithms do not invest effort during learning in abstract- ing from the tr-lnlng data, such as eager-learning (e.g., decision-tree algorithms, rule-induction, or connectionist-learning algorithms, (Qululan, 1993; Mitchell, 1997)) do. Rather, they defer investing effort until new instances axe presented. On be- ing presented with an instance, a memory-based *This research was done in the context of the "Induc- tion of Linguistic Knowledge" research programme, par- tially supported by the Foundation for Language Speech and Logic (TSL), which is funded by the Netherlands Organization for Scientific Research (NWO). Part of the first author's work was performed at the Department of Computer Science of the Unlversiteit Maastricht. learning algorithm searches for a best-matching in- stance, or, more generically, a set of the k best- matching instances in memory. Having found such a set of h best-matching instances, the algorithm takes the (majority) class with which the instances in the set axe labeled to be the class of the new instance. Pure memory-based learning algorithms implement the classic k-nearest neighbour algo- rithm (Cover and Hart, 1967; Devijver and Kittler, 1982; Aha, Kibler, and Albert, 1991); in different contexts, memory-based learning algorithms have also been named lazy, instance-based, exemplar- based, memory-based, case-based learning or reason- ing (Stanfdl and Waltz, 1986; Kolodner, 1993; Aha, Kibler, and Albert, 1991; Aha, 1997)) Memory-based learning has been demonstrated to yield accurate models of various natural lan- guage tasks such as grapheme-phoneme conver- sion, word stress assignment, part-of-speech tagging, and PP-attachment (Daelemans, Van den Bosch, and Weijters, 1997a). For example, the memory- based learning algorithm ml-IG (Daelemans and Van den Bosch, 1992; Daclemans, Van den Bosch, and We~jters, 1997b), which extends the well-known …

Content truncated to 500 words