Clustering Semantically Equivalent Words into Cognate Sets in Multilingual Lists

Proceedings of the 5th International Joint Conference on Natural Language Processing, pages 865–873, Chiang Mai, Thailand, November 8 – 13, 2011. c⃝2011 AFNLP Clustering Semantically Equivalent Words into Cognate Sets in Multilingual Lists Bradley Hauer and Grzegorz Kondrak Department of Computing Science University of Alberta Edmonton, Alberta, Canada, T6G 2E8 {bmhauer,gkondrak}@ualberta.ca Abstract Word lists have become available for most of the world’s languages, but only a small fraction of such lists contain cognate in- formation. We present a machine-learning approach that automatically clusters words in multilingual word lists into cognate sets. Our method incorporates a number of di- verse word similarity measures and fea- tures that encode the degree of affinity be- tween pairs of languages. The output of the classification algorithm is then used to generate cognate groups. The results of the experiments on word lists representing several language families demonstrate the utility of the proposed approach. 1 Introduction Cognates are words with a shared linguistic origin, such as English father and French p`ere. Identifi- cation of cognates is essential in historical linguis- tics, and cognate information has been success- fully applied to natural language processing tasks, such as sentence alignment in bitexts (Simard et al., 1993), and statistical machine transla- tion (Kondrak et al., 2003). The problem of au- tomatically identifying pairs of cognates has been addressed previously (Frunza and Inkpen, 2009; Kondrak, 2009). The process of identification is usually based on the combination of the following three types of evidence: phonetic similarity, se- mantic similarity, and recurrent sound correspon- dences. The input data include dictionaries, mul- tilingual word lists, and bitexts. The objective can be finding pairs of cognates among two related languages, or finding groups of cognates among multiple languages. In this paper, we focus on the task of identi- fying cognate groups (clusters) in word lists on the basis of word similarity and language relation- ships. Word lists are now available for most of the world’s languages (Wichmann et al., 2011). How- ever, only a fraction of such lists contain cognate information. Methods proposed for pairwise cog- nate identification are of limited utility for such data because they fail to consider the transitivity property of the cognation relationship. Cognate groups are also more useful than cognate pairs as the input to algorithms for reconstructing phyloge- netic trees (Bouchard-Cˆot´e et al., 2007). A number of word similarity measures have been applied to the problem of cognate identifi- cation (Frunza et al., 2005). Kondrak and Dorr (2004) report that a simple average of various measures outperforms any individual measure of phonetic similarity. We propose to combine mea- sures using a machine-learning approach based on support vector machines (SVMs). The SVMs are trained on both positive and negative examples, and allow for a seamless combination of a num- ber of diverse similarity measures. In addition to word similarity features, we in- clude a set of language-pair features that incor- porate information regarding the degree of relat- edness between languages. We develop a way to self-train these features in …

Content truncated to 500 words