Combining Information-Weighted Sequence Alignment and Sound Correspondence Models for Improved Cognate Detection

Proceedings of the 27th International Conference on Computational Linguistics, pages 3123–3133 Santa Fe, New Mexico, USA, August 20-26, 2018. 3123 Combining Information-Weighted Sequence Alignment and Sound Correspondence Models for Improved Cognate Detection Johannes Dellert Seminar f¨ur Sprachwissenschaft Universit¨at T¨ubingen johannes.dellert@uni-tuebingen.de Abstract Methods for automated cognate detection in historical linguistics invariably build on some mea- sure of form similarity which is designed to capture the remaining systematic similarities between cognate word forms after thousands of years of divergence. A wide range of clustering and clas- sification algorithms has been explored for the purpose, whereas possible improvements on the level of pairwise form similarity measures have not been the main focus of research. The ap- proach presented in this paper improves on this core component of cognate detection systems by a novel combination of information weighting, a technique for putting less weight on reoccur- ring morphological material, with sound correspondence modeling by means of pointwise mu- tual information. In evaluations on expert cognacy judgments over a subset of the IPA-encoded NorthEuraLex database, the combination of both techniques is shown to lead to considerable improvements in average precision for binary cognate detection, and modest improvements for distance-based cognate clustering. 1 Introduction The inference of cognate sets, i.e. sets of words which are related by being inherited from a common ancestor, is a central problem of historical linguistics. In the classical comparative method, the detection of cognates is an elementary preprocessing step for establishing sound laws, reconstructing the original forms in proto-languages, and thereby ultimately showing that groups of languages are related by com- mon ancestry. In computational historical linguistics, cognacy-encoded datasets are the most common input format for modern methods of phylogenetic inference as pioneered by Gray and Jordan (2000). While in classical historical linguistics, a sharp distinction is made between cognates connected purely by inheritance, and words which are related due to borrowing, the common usage in the computational field refers to both types of relations as cognacy, with the understanding that the primary distinction is whether words are etymologically related, and the decision whether they are related by inheritance or borrowing will be made at a later refinement stage. Since this paper deals with automated cognate detection, the term will be used in the broader sense. This paper builds on Dellert and Buch (to appear), where we introduced a segment-wise information content model as a technique for alignment and comparison of IPA strings. Information content is used to focus the string distance judgment on the parts of the word forms that are distinctive with respect to all the other words in the language. The main benefit for cognate detection is that it weakens distorting effects older approaches were faced with when dealing with lexical forms that were not reduced to stems. In this paper, I take the next step of combining information weighting with the more established idea of inferring specialized segment similarity matrices for each language pair, with the goal of capturing some of the regularity in sound correspondences caused …

Content truncated to 500 words