Linguistically Naïve != Language Independent: Why NLP Needs Linguistic Typology

Proceedings of the EACL 2009 Workshop on the Interaction between Linguistics and Computational Linguistics, pages 26–32, Athens, Greece, 30 March, 2009. c⃝2009 Association for Computational Linguistics Linguistically Na¨ıve != Language Independent: Why NLP Needs Linguistic Typology Emily M. Bender University of Washington Seattle, WA, USA ebender@u.washington.edu Abstract In this position paper, I argue that in order to create truly language-independent NLP systems, we need to incorporate linguis- tic knowledge. The linguistic knowledge in question is not intricate rule systems, but generalizations from linguistic typol- ogy about the range of variation in linguis- tic structures across languages. 1 Introduction Language independence is commonly presented as one of the advantages of modern, machine- learning approaches to NLP. Once an algorithm is developed, the argument goes, it can trivially be extended to another language; “all” that is needed is a suitably large amount of training data for the new language.1 This is indeed a virtue. How- ever, the typical approach to developing language- independent systems is to eschew using any lin- guistic knowledge in their production. In this po- sition paper, I argue that, on the contrary, the pro- duction of language-independent NLP technology requires linguistic knowledge, and that the rele- vant kind of linguistic knowledge is in fact rela- tively inexpensive. The rest of this paper is structured as follows: In Section 2, I discuss how linguistically na¨ıve systems can end up tuned to the languages they were originally developed for. In Section 3, I survey the long papers from ACL2008:HLT to give a snapshot of how linguistic diversity is cur- rently handled in our field. In Section 4, I give 1This of course abstracts away from the production of such data, which may require both significant pre-processing and annotation work. For the purposes of the present argu- ment, however, we can assume that all language-independent NLP systems are unicode-enabled, assume a definition of “word” that is cross-linguistically applicable, and require the type of annotations that are likely to have already been de- ployed for another purpose. a brief overview of Linguistic Typology, and sug- gest how knowledge derived from this field can be profitably incorporated into language-independent NLP systems. 2 Hidden Language Dependence A simple example of subtle language dependence is the way in which n-gram models work better for languages that share important typological proper- ties with English. On the face of it, n-gram mod- els code in no linguistic knowledge. They treat natural language text as simple sequences of sym- bols and automatically reflect the “hidden” struc- ture through the way it affects the distributions of words in various (flat, unstructured) contexts. However, the effectiveness of n-gram models in English (and similar languages) is partially pred- icated on two properties of those languages: rel- atively low levels of inflectional morphology, and relatively fixed word order. As is well-known by now, languages with more elaborate morphology (more morphemes per word, more distinctions within the same number of morphological slots, and/or fewer uninflected words) present greater data sparsity problems for language …

Content truncated to 500 words