Language-independent compound splitting with morphological operations

Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics, pages 1395–1404, Portland, Oregon, June 19-24, 2011. c⃝2011 Association for Computational Linguistics Language-independent Compound Splitting with Morphological Operations Klaus Macherey1 Andrew M. Dai2 David Talbot1 Ashok C. Popat1 Franz Och1 1Google Inc. 1600 Amphitheatre Pkwy. Mountain View, CA 94043, USA {kmach,talbot,popat,och}@google.com 2University of Edinburgh 10 Crichton Street Edinburgh, UK EH8 9AB a.dai@ed.ac.uk Abstract Translating compounds is an important prob- lem in machine translation. Since many com- pounds have not been observed during train- ing, they pose a challenge for translation sys- tems. Previous decompounding methods have often been restricted to a small set of lan- guages as they cannot deal with more complex compound forming processes. We present a novel and unsupervised method to learn the compound parts and morphological operations needed to split compounds into their com- pound parts. The method uses a bilingual corpus to learn the morphological operations required to split a compound into its parts. Furthermore, monolingual corpora are used to learn and filter the set of compound part can- didates. We evaluate our method within a ma- chine translation task and show significant im- provements for various languages to show the versatility of the approach. 1 Introduction A compound is a lexeme that consists of more than one stem. Informally, a compound is a combina- tion of two or more words that function as a single unit of meaning. Some compounds are written as space-separated words, which are called open com- pounds (e.g. hard drive), while others are written as single words, which are called closed compounds (e.g. wallpaper). In this paper, we shall focus only on closed compounds because open compounds do not require further splitting. The objective of compound splitting is to split a compound into its corresponding sequence of con- stituents. If we look at how compounds are created from lexemes in the first place, we find that for some languages, compounds are formed by concatenating existing words, while in other languages compound- ing additionally involves certain morphological op- erations. These morphological operations can be- come very complex as we illustrate in the following case studies. 1.1 Case Studies Below, we look at splitting compounds from 3 differ- ent languages. The examples introduce in part the notation used for the decision rule outlined in Sec- tion 3.1. 1.1.1 English Compound Splitting The word flowerpot can appear as a closed or open compound in English texts. To automatically split the closed form we have to try out every split point and choose the split with minimal costs according to a cost function. Let's assume that we already know that flowerpot must be split into two parts. Then we have to position two split points that mark the end of each part (one is always reserved for the last charac- ter position). The number of split points is denoted by K (i.e. K = 2), while the position of split points is denoted by n1 and n2. Since flowerpot consists of 9 …

Content truncated to 500 words