Statistical Machine Translation of Australian Aboriginal Languages: Morphological Analysis with Languages of Differing Morphological Richness

Statistical Machine Translation of Australian Aboriginal Languages: Morphological Analysis with Languages of Differing Morphological Richness Simon Zwarts Centre for Language Technology Macquarie University Sydney, Australia szwarts@ics.mq.edu.au Mark Dras Centre for Language Technology Macquarie University Sydney, Australia madras@ics.mq.edu.au Abstract Morphological analysis is often used during preprocessing in Statistical Machine Trans- lation. Existing work suggests that the bene- fit would be greater for more highly inflected languages, although to our knowledge this has not been systematically tested on lan- guages with comparable morphology. In this paper, two comparable languages with different amounts of inflection are tested, to see if the benefits of morphology used during the translation process, depends on the morphological richness of the language. For this work we use indigenous Australian languages: most Australian Aboriginal lan- guages are highly inflected, where words can take a considerable number of post- fixes when compared to Indo-European lan- guages, and for languages in the same (Pama Nyungan) family, the morphological sys- tem works similarly. We show in this pre- liminary work that morphological analysis clearly benefits the richer of the two lan- guages investigated, but is more equivocal in the case of the other. 1 Introduction The majority of research in the field of Machine Translation (MT) nowadays takes a statistical ap- proach. Morphologically rich languages have some characteristics which make MT hard, particularly in the statistical MT (SMT) context. In one common language group we want to investigate the effect of applying special morphological treatment within SMT for languages with varying degree of morpho- logical richness. Without any morphological pre- processing, individual word counts can be quite low in highly inflected languages, causing more data sparseness than necessary, and ignoring some infor- mation which might be useful in Natural Language Processing. Preprocessing before SMT has been used as a way of improving results. This ranges from basic tokeni- sation (e.g. separating possessive ’s on English be- fore training) to extensive syntax-based reordering (e.g. Collins et al. (2005)). Often, the choice of preprocessing proceeds without consideration of the type of language; consider for example recent work on Arabic (Sadat and Habash, 2006), where the var- ious combinations of different preprocessing strate- gies are systematically worked through, with no par- ticular attention to the characteristics of Arabic. In most work, there is an intuitive notion that there is a connection between morphological richness of a language and the usefulness of morphological pre- processing. This is suggested in its use in pars- ing for Korean (Han and Sarkar, 2002) and Turkish (Eryi˘git and Oflazer, 2006), and MT for Czech (Al- Onaizan et al., 1999). But in this body of work, as well as the body of work mentioned in section 3.1, only analysis of one language is performed. More- over there is no specific measure of richness of mor- phology; it is not obvious how to compare the mor- phology of different languages such as English, Ara- bic, Turkish or Korean with their different combina- tions of prefixing, suffixing and infixing. In this pa- per, …

Content truncated to 500 words