Using Adaptor Grammars to Identify Synergies in the Unsupervised Acquisition of Linguistic Structure

Proceedings of ACL-08: HLT, pages 398–406, Columbus, Ohio, USA, June 2008. c⃝2008 Association for Computational Linguistics Using adaptor grammars to identify synergies in the unsupervised acquisition of linguistic structure Mark Johnson Brown University Mark Johnson@Brown.edu Abstract Adaptor grammars (Johnson et al., 2007b) are a non-parametric Bayesian extension of Prob- abilistic Context-Free Grammars (PCFGs) which in effect learn the probabilities of en- tire subtrees. In practice, this means that an adaptor grammar learns the structures useful for generating the training data as well as their probabilities. We present several differ- ent adaptor grammars that learn to segment phonemic input into words by modeling dif- ferent linguistic properties of the input. One of the advantages of a grammar-based frame- work is that it is easy to combine grammars, and we use this ability to compare models that capture different kinds of linguistic structure. We show that incorporating both unsupervised syllabification and collocation-finding into the adaptor grammar significantly improves un- supervised word-segmentation accuracy over that achieved by adaptor grammars that model only one of these linguistic phenomena. 1 Introduction How humans acquire language is arguably the cen- tral issue in the scientific study of language. Hu- man language is richly structured, but it is still hotly debated as to whether this structure can be learnt, or whether it must be innately specified. Compu- tational linguistics can contribute to this debate by identifying which aspects of language can poten- tially be learnt from the input available to a child. Here we try to identify linguistic properties that convey information useful for learning to segment streams of phonemes into words. We show that si- multaneously learning syllable structure and collo- cations improves word segmentation accuracy com- pared to models that learn these independently. This suggests that there might be a synergistic interaction in learning several aspects of linguistic structure si- multaneously, as compared to learning each kind of linguistic structure independently. Because learning collocations and word-initial syllable onset clusters requires the learner to be able to identify word boundaries, it might seem that we face a chicken-and-egg problem here. One of the im- portant properties of the adaptor grammar inference procedure is that it gives us a way of learning these interacting linguistic structures simultaneously. Adaptor grammars are also interesting because they can be viewed as directly inferring linguistic structure. Most well-known machine-learning and statistical inference procedures are parameter esti- mation procedures, i.e., the procedure is designed to find the values of a finite vector of parameters. Stan- dard methods for learning linguistic structure typi- cally try to reduce structure learning to parameter estimation, say, by using an iterative generate-and- prune procedure in which each iteration consists of a rule generation step that proposes new rules ac- cording to some scheme, a parameter estimation step that estimates the utility of these rules, and pruning step that removes low utility rules. For example, the Bayesian unsupervised PCFG estimation procedure devised by Stolcke (1994) uses a model-merging procedure to propose new sets of PCFG rules and a …

Content truncated to 500 words