Language Acquisition and Probabilistic Models: keeping it simple

Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, pages 1321–1330, Sofia, Bulgaria, August 4-9 2013. c⃝2013 Association for Computational Linguistics Language Acquisition and Probabilistic Models: keeping it simple Aline Villavicencio♣, Marco Idiart♥ Robert Berwick♦, Igor Malioutov♠ ♣Institute of Informatics, Federal University of Rio Grande do Sul (Brazil) ♥Institute of Physics, Federal University of Rio Grande do Sul (Brazil) ♦LIDS, Dept. of EECS, Massachusetts Institute of Technology (USA) ♠ CSAIL, Dept. of EECS, Massachusetts Institute of Technology (USA) avillavicencio@inf.ufrgs.br, marco.idiart@if.ufrgs.br berwick@csail.mit.edu, igorm@mit.edu Abstract Hierarchical Bayesian Models (HBMs) have been used with some success to capture empirically observed pat- terns of under- and overgeneralization in child language acquisition. How- ever, as is well known, HBMs are “ideal” learning systems, assuming ac- cess to unlimited computational re- sources that may not be available to child language learners. Conse- quently, it remains crucial to carefully assess the use of HBMs along with al- ternative, possibly simpler, candidate models. This paper presents such an evaluation for a language acquisi- tion domain where explicit HBMs have been proposed: the acquisition of En- glish dative constructions. In particu- lar, we present a detailed, empirically- grounded model-selection compari- son of HBMs vs. a simpler alternative based on clustering along with max- imum likelihood estimation that we call linear competition learning (LCL). Our results demonstrate that LCL can match HBM model performance with- out incurring on the high computa- tional costs associated with HBMs. 1 Introduction In recent years, with advances in probability and estimation theory, there has been much interest in Bayesian models (BMs) (Chater, Tenenbaum, and Yuille, 2006; Jones and Love, 2011) and their application to child lan- guage acquisition with its challenging com- bination of structured information and in- complete knowledge, (Perfors, Tenenbaum, and Wonnacott, 2010; Hsu and Chater, 2010; Parisien, Fazly, and Stevenson, 2008; Parisien and Stevenson, 2010) as they offer several ad- vantages in this domain. They can readily handle the evident noise and ambiguity of ac- quisition input, while at the same time pro- viding efficiency via priors that mirror known pre-existing language biases. Further, hierar- chical Bayesian Models (HBMs) can combine distinct abstraction levels of linguistic knowl- edge, from variation at the level of individ- ual lexical items, to cross-item variation, using hyper-parameters to capture observed pat- terns of both under- and over-generalization as in the acquisition of e.g. dative alterna- tions in English (Hsu and Chater, 2010; Per- fors, Tenenbaum, and Wonnacott, 2010), and verb frames in a controlled artificial language (Wonnacott, Newport, and Tanenhaus, 2008). HBMs can thus be viewed as providing a “rational” upper bound on language learn- ability, yielding optimal models that account for observed data while minimizing any re- quired prior information. In addition, the clustering implicit in HBM modeling intro- duces additional parameters that can be tuned to specific data patterns. However, this comes at a well-known price: HBMs generally are also ideal learning systems, known to be computationally infeasible (Kwisthout, Ware- ham, and van Rooij, 2011). Approximations proposed to ensure computational …

Content truncated to 500 words