Memory-Based Learning of Morphology with Stochastic Transducers

Memory-Based Learning of Morphology with Stochastic Transducers Alexander Clark ISSCO / TIM University of Geneva UNI-MAIL, Boulevard du Pont-d’Arve, CH-1211 Gen`eve 4, Switzerland Alex.Clark@issco.unige.ch Abstract This paper discusses the supervised learn- ing of morphology using stochastic trans- ducers, trained using the Expectation- Maximization (EM) algorithm. Two ap- proaches are presented: first, using the transducers directly to model the process, and secondly using them to define a sim- ilarity measure, related to the Fisher ker- nel method (Jaakkola and Haussler, 1998), and then using a Memory-Based Learn- ing (MBL) technique. These are evaluated and compared on data sets from English, German, Slovene and Arabic. 1 Introduction Finite-state methods are in large part adequate to model morphological processes in many languages. A standard methodology is that of two-level mor- phology (Koskenniemi, 1983) which is capable of handling the complexity of Finnish, though it needs substantial extensions to handle non-concatenative languages such as Arabic (Kiraz, 1994). These mod- els are primarily concerned with the mapping from deep lexical strings to surface strings, and within this framework learning is in general difficult (Itai, 1994). In this paper I present algorithms for learn- ing the finite-state transduction between pairs of un- inflected and inflected words. – supervised learning of morphology. The techniques presented here are, however, applicable to learning other types of string transductions. Memory-based techniques, based on principles of non-parametric density estimation, are a powerful form of machine learning well-suited to natural lan- guage tasks. A particular strength is their ability to model both general rules and specific exceptions in a single framework (van den Bosch and Daelemans, 1999). However they have generally only been used in supervised learning techniques where a class label or tag has been associated to each feature vector. Given these manual or semi-automatic class labels, a set of features and a pre-defined distance function new in- stances are classified according to the class label of the closest instance. However these approaches are not a complete solution to the problem of learning morphology, since they do not directly produce the transduction. The problem must first be converted into an appropriate feature-based representation and classified in some way. The techniques presented here operate directly on sequences of atomic sym- bols, using a much less articulated representation, and much less input information. 2 Stochastic Transducers It is possible to apply the EM algorithm to learn the parameters of stochastic transducers, (Ristad, 1997; Casacuberta, 1995; Clark, 2001a). (Clark, 2001a) showed how this approach could be used to learn morphology by starting with a randomly initialized model and using the EM algorithm to find a local maximum of the joint probabilities over the pairs of inflected and uninflected words. In addition rather than using the EM algorithm to optimize the joint probability it would be possible to use a gradient de- Computational Linguistics (ACL), Philadelphia, July 2002, pp. 513-520. Proceedings of the 40th Annual Meeting of the Association for scent algorithm to maximize the conditional proba- bility. The models used here are Stochastic Non- …

Content truncated to 500 words