LexStat: Automatic Detection of Cognates in Multilingual Wordlists

Proceedings of the EACL 2012 Joint Workshop of LINGVIS & UNCLH, pages 117–125, Avignon, France, April 23 - 24 2012. c⃝2012 Association for Computational Linguistics LexStat: Automatic Detection of Cognates in Multilingual Wordlists Johann-Mattis List Institute for Romance Languages and Literature Heinrich Heine University Düsseldorf, Germany listm@phil.uni-duesseldorf.de Abstract In this paper, a new method for automatic cognate detection in multilingual wordlists will be presented. The main idea behind the method is to combine different approaches to sequence comparison in historical linguistics and evolutionary biology into a new frame- work which closely models the most impor- tant aspects of the comparative method. The method is implemented as a Python program and provides a convenient tool which is pub- licly available, easily applicable, and open for further testing and improvement. Testing the method on a large gold standard of IPA- encoded wordlists showed that its results are highly consistent and outperform previous methods. 1 Introduction During the last two decades there has been an in- creasing interest in automatic approaches to his- torical linguistics, which is reflected in the large amount of literature on phylogenetic reconstruc- tion (e.g. Ringe et al., 2002; Gray and Atkin- son, 2003; Brown et al., 2008), statistical aspects of genetic relationship (e.g. Baxter and Manaster Ramer, 2000; Kessler, 2001; Mortarino, 2009), and phonetic alignment (e.g. Kondrak, 2002; Prokić et al., 2009; List, forthcoming). While the supporters of these new automatic methods would certainly agree that their greatest advantage lies in the increase of repeatability and objectivity, it is interesting to note that the most crucial part of the analysis, namely the identifica- tion of cognates in lexicostatistical datasets, is still almost exclusively carried out manually. That this may be problematic was recently shown in a com- parison of two large lexicostatistical datasets pro- duced by different scholarly teams where differ- ences in item translation and cognate judgments led to topological differences of 30% and more (Geisler and List, forthcoming). Unfortunately, automatic approaches to cognate detection still lack the precision of trained linguists’ judgments. Furthermore, most of the methods that have been proposed so far only deal with bilingual as op- posed to multilingual wordlists. The LexStat method, which will be presented in the following, is a convenient tool which not only closely renders the most important aspects of manual approaches but also yields transparent de- cisions that can be directly compared with the re- sults achieved by the traditional methods. 2 Identification of Cognates 2.1 The Comparative Method In historical linguistics, cognacy is traditionally determined within the framework of the compar- ative method (Trask, 2000, 64-67). The final goal of this method is the reconstruction of proto- languages, yet the basis of the reconstruction it- self rests on the identification of cognate words or morphemes within genetically related languages. Within the comparative method, cognates in a given set of language varieties are identified by applying a recursive procedure. First an initial list of putative cognate sets is created by comparing semantically and phonetically similar words from the …

Content truncated to 500 words