Cross-Entropy and Linguistic Typology

I II l / / l / / / / / Cross-Entropy and Linguistic Typology /. Patrick Juola Department of Experimental Psychology University of Oxford Oxford, UK OX1 3UD patrick, j uola@psy, ox. ac. uk Abstract The idea of '~familial relationships" among lan- guages is well-established and accepted, al- though some controversies persist in a few specific instances. By painstakingly record- ing and identifying regularities and similarities and comparing these to the historical record, linguists have been able to produce a general "family tree" incorporating most natural lan- guages. We suggest here that much of these trees can be automatically determined by a complemen- tary technique of distributional analysis. Re- cent work by (Farach et al., 1995) and (Juola, 1997) suggests that Kullback-Leibler diver- gence (or cross-entropy) can be meaningfully measured from small samples, in some cases as small as only 20 or so words. Using these techniques, we define and measure a distance function between translations of a small corpus (c. 70 words/sample) covering much of the ac- cepted Indo-European family, and reconstruct a relationship tree by hierarchical duster anal- ysis. The resulting tree shows remarkable sim- ilarity to the accepted Indo-European family; this we read as evidence both for the immense power of this measurement technique and for the validity of this kind of mechanical similar- ity judgement in the identification of typologi- cal relationships. Furthermore, this technique is in theory sensitive to different sorts of rela- tionships than more common word-list based methods and may help ilium;hate these from a different direction. 1 Introduction Over the past century, a large amount of research effort has gone into the establishment of structures describing the typological and taxonomic relation- ships among languages past and present; the well- known "Romance language" group, consisting of all the languages in some sense "descended from" Latin is an example. In addition to their inherent interest, the results of these studies can be of use in telling us about the relationships, cultures, and environments of people and tribes long-distant from our present world. Although these techniques are powerful, they are limited in their application in several ways. The traditional focus on word lists as the primary tool for language classification excludes syntax and mor- phology from consideration. By constructing these word lists out of only basic lexical items, the appli- cability is further limited. Although in theory these problems could be avoided by simply constructing different lists, there is still a problem with the vol- ume of data to be processed -- if the comparisons are performed at the level of "language," it is dif- ficult if not impossible to discuss questions such as whether "legal English" shows more French influence than "standard English" or vice versa. However, the answers (were they available) to questions like this could be useful to, for example, socioliaguists in attempting to trace the relationships between and among subgroups within a culture. The results presented in this paper suggest that distributional analyses can provide much of the same sort …

Content truncated to 500 words