Word classes in Indonesian: A linguistic reality or a convenient fallacy in natural language processing?

Word classes in Indonesian: A linguistic reality or a convenient fallacy in natural language processing? Meladel Mistica a, Timothy Baldwin b, and I Wayan Arka a aCHL, The Australian National University, Canberra, Australia meladel.mistica@gmail.com wayan.arka@anu.edu.au bCSSE, The University of Melbourne, Parkville, Australia tb@ldwin.net Abstract. This paper looks at Indonesian (Bahasa Indonesia), and the claim that there is no noun-verb distinction within the language as it is spoken in regions such as Riau and Jakarta. We test this claim for the language as it is written by a variety of Indonesian speakers using empirical methods traditionally used in part-of-speech induction. In this study we use only morphological patterns that we generate from a pre-existing mor- phological analyser. We find that once the distribution of the data points in our experiments match the distribution of the text from which we gather our data, we obtain significant re- sults that show a distinction between the class of nouns and the class of verbs in Indonesian. Furthermore it shows promise that the labelling of word classes may be achieved only with morphological features, which could be applied to out-of-vocabulary items. Keywords: Indonesian, Word class, Word class induction, Morphology 1 Introduction The notion of word classes, such as the nouns, verbs and adjectives, is fundamental in both lin- guistics and computational linguistics. Word classes are the basis for the labels in part-of-speech tagging, and also the building blocks for parsing. In grammar engineering, they are the primitives upon which context-free grammar rules are written. In linguistics, they are considered the cate- gories that shape the organisation of the language, and the way the world is conceived through language. These categories may not align across languages: what is expressed as a verb in one language may be expressed as an adjective or noun in another. But one linguistic universality hypothesis that remains despite these variations is that the categories noun and verb exist in all languages (Croft, 2003). This paper examines the noun–verb distinction specifically for Indonesian (Bahasa Indonesia) as, at least in spoken Indonesian in particular regions such as Riau and Jakarta, it has been claimed that open class categories are indistinguishable (Gil, 2001; Gil, 2010). If correct, this would refute the universality of the noun–verb distinction in Linguistics, with significant implications for Linguistic Typology and Theoretical Syntax. The primary aim in this paper is to apply unsupervised data-driven analysis of Indonesian text to determine whether we can automatically learn the noun–verb distinction, and in doing so, shed light on whether Indonesian conforms to Croft’s noun–verb universality hypothesis, or is indeed a counter-example to the hypothesis as claimed by Gil. In addition, these experiments are a litmus test to see if “morphological signatures”, as used by Goldsmith (2001), are sufficient in Copyright 2011 by Meladel Mistica, Timothy Baldwin, and I Wayan Arka 25th Pacific Asia Conference on Language, Information and Computation, pages 293–302 293 determining parts-of-speech, and can be used to determine the part-of-speech of out-of-vocabulary items (OOV) in Indonesian text processing. One assumption we …

Content truncated to 500 words