An Incremental Bayesian Model for Learning Syntactic Categories

CoNLL 2008: Proceedings of the 12th Conference on Computational Natural Language Learning, pages 89–96 Manchester, August 2008 An Incremental Bayesian Model for Learning Syntactic Categories Christopher Parisien, Afsaneh Fazly and Suzanne Stevenson Department of Computer Science University of Toronto Toronto, ON, Canada [chris,afsaneh,suzanne]@cs.toronto.edu Abstract We present an incremental Bayesian model for the unsupervised learning of syntactic cate- gories from raw text. The model draws infor- mation from the distributional cues of words within an utterance, while explicitly bootstrap- ping its development on its own partially- learned knowledge of syntactic categories. Testing our model on actual child-directed data, we demonstrate that it is robust to noise, learns reasonable categories, manages lexical ambiguity, and in general shows learning be- haviours similar to those observed in children. 1 Introduction An important open problem in cognitive science and artificial intelligence is how children successfully learn their native language despite the lack of explicit training. A key challenge in the early stages of lan- guage acquisition is to learn the notion of abstract syntactic categories (e.g., nouns, verbs, or determin- ers), which is necessary for acquiring the syntactic structure of language. Indeed, children as young as two years old show evidence of having acquired a good knowledge of some of these abstract categories (Olguin and Tomasello, 1993); by around six years of age, they have learned almost all syntactic categories (Kemp et al., 2005). Computational models help to elucidate the kinds of learning mechanisms that may be capable of achieving this feat. Such studies shed light on the possible cognitive mechanisms at work in human language acquisition, and also on potential means for unsupervised learning of complex linguis- tic knowledge in a computational system. Learning the syntactic categories of words has been suggested to be based on the morphological and phonological properties of individual words, as well c⃝ 2008. Licensed under the Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported li- cense (http://creativecommons.org/licenses/by-nc-sa/3.0/). Some rights reserved. as on the distributional information about the con- texts in which they appear. Several computational models have been proposed that draw on one or more of the above-mentioned properties in order to group words into discrete unlabeled categories. Most ex- isting models only intend to show the relevance of such properties to the acquisition of adult-like syn- tactic categories such as nouns and verbs; hence, they do not necessarily incorporate the types of learning mechanisms used by children (Sch¨utze, 1993; Red- ington et al., 1998; Clark, 2000; Mintz, 2003; Onnis and Christiansen, 2005). For example, in contrast to the above models, children acquire their knowledge of syntactic categories incrementally, processing the utterances they hear one at a time. Moreover, chil- dren appear to be sensitive to the fact that syntactic categories are partially defined in terms of other cat- egories, e.g., nouns tend to follow determiners, and can be modified by adjectives. We thus argue that a computational model should be incremental, and should use more abstract cate- gory knowledge to help better identify syntactic cat- egories. Incremental processing also allows …

Content truncated to 500 words