How the Statistical Revolution Changes (Computational) Linguistics

Proceedings of the EACL 2009 Workshop on the Interaction between Linguistics and Computational Linguistics, pages 3–11, Athens, Greece, 30 March, 2009. c⃝2009 Association for Computational Linguistics How the statistical revolution changes (computational) linguistics Mark Johnson Cognitive and Linguistic Sciences and Computer Science Brown University Mark Johnson@Brown.edu Abstract This paper discusses some of the ways that the “statistical revolution” has changed and continues to change the relationship between linguistics and computational lin- guistics. I claim that it is more useful in parsing to make an open world assumption about possible linguistic structures, rather than the closed world assumption usu- ally made in grammar-based approaches to parsing, and I sketch two different ways in which grammar-based approaches might be modified to achieve this. I also de- scribe some of the ways in which proba- bilistic models are starting to have a sig- nificant impact on psycholinguistics and language acquisition. In language acqui- sition Bayesian techniques may let us em- pirically evaluate the role of putative uni- versals in universal grammar. 1 Introduction The workshop organizers asked us to write some- thing controversial to stimulate discussion, and I’ve attempted to do that here. Usually in my pa- pers I try to stick to facts and claims that I can sup- port, but here I have fearlessly and perhaps fool- ishly gone out on a limb and presented guesses, hunches and opinions. Take them with a grain of salt. Inspired by Wanamaker’s well-known quote about advertising, I expect that half of the ideas I’m proposing here are wrong, but I don’t know which half. I hope the conference will help me figure that out. Statistical techniques have revolutionized many scientific fields in the past two decades, including computational linguistics. This paper discusses the impact of this on the relationship between computational linguistics and linguistics. I’m pre- senting a personal perspective rather than a scien- tific review here, and for this reason I focus on ar- eas I have some experience with. I begin by dis- cussing how the statistical perspective changed my understanding of the relationship between linguis- tic theory, grammars and parsing, and then go on to describe some of the ways that ideas from statis- tics and machine learning are starting to have an impact on linguistics today. Before beginning, I’d like to say something about what I think computational linguistics is. I view computational linguistics as having both a scientific and an engineering side. The engineer- ing side of computational linguistics, often called natural language processing (NLP), is largely con- cerned with building computational tools that do useful things with language, e.g., machine trans- lation, summarization, question-answering, etc. Like any engineering discipline, natural language processing draws on a variety of different scien- tific disciplines. I think it’s fair to say that in the current state of the art, natural language processing draws far more heavily on statistics and machine learning than it does on linguistic theory. For example, one might claim that all an NLP engineer really needs to understand about …

Content truncated to 500 words