A Novel Evaluation Method for Morphological Segmentation

A Novel Evaluation Method for Morphological Segmentation Javad Nouri, Roman Yangarber Department of Computer Science University of Helsinki, Finland First.Last@cs.helsinki.fi Abstract Unsupervised learning of morphological segmentation of words in a language, based only on a large corpus of words, is a challenging task. Evaluation of the learned segmentations is a challenge in itself, due to the inherent ambiguity of the segmentation task. There is no way to posit unique “correct” segmentation for a set of data in an objective way. Two models may arrive at different ways of segmenting the data, which may nonetheless both be valid. Several evaluation methods have been proposed to date, but they do not insist on consistency of the evaluated model. We introduce a new evaluation methodology, which enforces correctness of segmentation boundaries while also assuring consistency of segmentation decisions across the corpus. Keywords: Morphology, unsupervised learning, segmentation, evaluation, consistency. 1. Introduction Over the last 15 years, a number of algorithms have been proposed for learning of the morphological structure of lan- guages from raw data—large lists of words in the language (the corpus). This paper addresses the problem of evaluat- ing the performance of such algorithms. Morphology learn- ing is especially interesting for morphologically rich lan- guages, where words can consist of many morphs, which are surface realizations of morphemes; a morpheme is a minimal unit, which carries meaning or syntactic function. Ultimately, the problem of morphological analysis includes several aspects. One is identifying the structure of each word—which segments it consists of, and the types of seg- ments. In many languages segments can be stems and af- fixes; affixes can be prefixes, suffixes, etc. Another aspect is identifying allomorphy, which involves linking different variants (allomorphs) of a given morpheme, which appear in different words, and typically depend on phonological context. For example, it is not trivial to discover that in English try-ing and tri-ed have the same stem morpheme. It is well understood how one can build morphological an- alyzers for many languages. This is typically done by writ- ing morphological grammars, based on dictionaries of mor- phemes for stems and affixes, and writing rules for combin- ing the morphemes into words, e.g., (Koskenniemi, 1983). The morphological analyzer is then used to analyze any word in text, and is an essential low-level component in many tasks in natural language processing (NLP), such as parsing, machine translation, etc. Unsupervised learning of morphology is the inverse prob- lem: to infer the morphology of all words in the language, given only a large number of words—given no dictionar- ies, rules, etc. A number of algorithms have been proposed and their performance has been compared, e.g., in a series of shared-task competitions, the Morpho-Challenges, (Ku- rimo et al., 2010). The competitions provided annotated data, and tools for evaluation of performance. The majority of algorithms for unsupervised learning of morphology (that we have reviewed), treat the segmenta- tion of words into morphs either as the ultimate goal or as a first step on the way toward learning the …

Content truncated to 500 words