Measuring Language Divergence by Intra-Lexical Comparison

Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the ACL, pages 273–280, Sydney, July 2006. c⃝2006 Association for Computational Linguistics Measuring Language Divergence by Intra-Lexical Comparison T. Mark Ellison Informatics University of Edinburgh mark@markellison.net Simon Kirby Language Evolution and Computation Research Unit Philosophy, Psychology and Language Sciences, University of Edinburgh simon@ling.ed.ac.uk Abstract This paper presents a method for build- ing genetic language taxonomies based on a new approach to comparing lexi- cal forms. Instead of comparing forms cross-linguistically, a matrix of language- internal similarities between forms is cal- culated. These matrices are then com- pared to give distances between languages. We argue that this coheres better with current thinking in linguistics and psy- cholinguistics. An implementation of this approach, called PHILOLOGICON, is de- scribed, along with its application to Dyen et al.’s (1992) ninety-five wordlists from Indo-European languages. 1 Introduction Recently, there has been burgeoning interest in the computational construction of genetic lan- guage taxonomies (Dyen et al., 1992; Nerbonne and Heeringa, 1997; Kondrak, 2002; Ringe et al., 2002; Benedetto et al., 2002; McMahon and McMahon, 2003; Gray and Atkinson, 2003; Nakleh et al., 2005). One common approach to building language taxonomies is to ascribe language-language dis- tances, and then use a generic algorithm to con- struct a tree which explains these distances as much as possible. Two questions arise with this approach. The first asks what aspects of lan- guages are important in measuring inter-language distance. The second asks how to measure dis- tance given these aspects. A more traditional approach to building lan- guage taxonomies (Dyen et al., 1992) answers these questions in terms of cognates. A word in language A is said to be cognate with word in lan- guage B if the forms shared a common ancestor in the parent language of A and B. In the cognate- counting method, inter-language distance depends on the lexical forms of the languages. The dis- tance between two languages is a function of the number or fraction of these forms which are cog- nate between the two languages1. This approach to building language taxonomies is hard to imple- ment in toto because constructing ancestor forms is not easily automatable. More recent approaches, such as Kondrak’s (2002) and Heggarty et al’s (2005) work on di- alect comparison, take the synchronic word forms themselves as the language aspect to be compared. Variations on edit distance (see Kessler (2005) for a survey) are then used to evaluate differences be- tween languages for each word, and these differ- ences are aggregated to give a distance between languages or dialects as a whole. This approach is largely automatable, although some methods do require human intervention. In this paper, we present novel answers to the two questions. The features of language we will compare are not sets of words or phonological forms. Instead we compare the similarities be- tween forms, expressed as confusion probabilities. The distribution of confusion probabilities in one language is called a lexical metric. Section 2 …

Content truncated to 500 words