English Morphological Analysis with Machine-learned Rules

English Morphological Analysis with Machine-learned Rules Xuri TANG Dept. Foreign Languages Wuhan University of Science and Engineering, 430073，Wuhan, P. R. China tangxuriyz@yahoo.com.cn Abstract. This paper expounds an algorithm for morphological analysis of English language. The algorithm consists of two closely related components: morphological rule learning and morphological analyzing. The morphological rules are obtained through statistical learning from wordlist, with particular morphological features of English language taken into consideration. The procedure of morphological analysis considers two types of ambiguities: intersectional ambiguity and combinatory ambiguity. The procedure also considers the order of wordform formation in the language. Experiment shows that the algorithm performs distinctively compared to other algorithms. Keyword: Morphological analysis; statistical learning; intersectional ambiguity; combinatory ambiguity; wordform formation order 1 Introduction Automatic Morphological Analysis (CMA) is still a much discussed topic in natural language processing.1 The objective of CMA is to understand the inner mechanism of word form formation in a language. A morphological analyzer can provide valuable information for other computer- based linguistic tasks such as lemmatization, syntactic parsing, machine translation, information retrieving, text clustering and many others. Goldsmith (2001) classifies the work in automatic morphological analysis into four categories. The classification is done with an emphasis on how the morphological rules are obtained. The first category identifies morpheme boundaries on the basis of the degree of predictability of the n+1st letter given the first n letters. The second employs n-gram grammar and high likelihood to obtain morpheme internal structure. The third category endeavors to discover rules through phonological relationships between pairs of related words. The fourth one seeks for an analysis for a language which is the most concise and consequently boils down to a set of rules for the language. Two factors are essential to achieve accurate automatic morphological analysis. One factor is the construction of a set of morphological rules and the other is the morphological analysis procedure. The absence or underperformance of either of them impairs the overall ability of the morphological analyzer. Thus in constructing an algorithm for the task, both factors should be addressed. Machine learning is a promising alternative to obtain morphological rules. This method can avoid problems such as costly human labor, rule inconsistency and can provide additional statistical information which can be used in morphological analysis procedure. Based on information type employed in the machine learning task, we can usefully obtain two classes: supervised learning and unsupervised learning. The class of supervised learning (Bosch, 1999; Wicentowski, 2004) makes use of lexical database with morphological information. A good example of such lexical database is CELEX2. The class of unsupervised learning (Goldsmith, 2001; Kurino et al, 2004; Creutz, 2005) 1 For example, in 2005, a challenge named Unsupervised Segmentation of Words into Morphemes – Challenge 2005 was held in Helsinki University of Technology, Finland. 2 CELEX is a lexical information database, developed by Burnage et al. The database contains information about lemma, wordform, abbreviation, corpus tagging etc. 35 uses only a wordlist, with or without information of frequency along the way. In regard …

Content truncated to 500 words