Challenge Dataset of Cognates and False Friend Pairs from Indian Languages

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 3096–3102 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 3096 Challenge Dataset of Cognates and False Friend Pairs from Indian Languages Diptesh Kanojia†,♣,⋆, Pushpak Bhattacharyya†, Malhar Kulkarni†, and Gholamreza Haffari⋆ †Indian Institute of Technology Bombay, India ♣IITB-Monash Research Academy, India ⋆Monash University, Australia †{diptesh, pb, malhar}@iitb.ac.in, ⋆gholamreza.haffari@monash.edu Abstract Cognates are present in multiple variants of the same text across different languages (e.g., “hund” in German and “hound” in English language mean “dog”). They pose a challenge to various Natural Language Processing (NLP) applications such as Machine Translation, Cross-lingual Sense Disambiguation, Computational Phylogenetics, and Information Retrieval. A possible solution to address this challenge is to identify cognates across language pairs. In this paper, we describe the creation of two cognate datasets for twelve Indian languages, namely Sanskrit, Hindi, Assamese, Oriya, Kannada, Gujarati, Tamil, Telugu, Punjabi, Bengali, Marathi, and Malayalam. We digitize the cognate data from an Indian language cognate dictionary and utilize linked Indian language Wordnets to generate cognate sets. Additionally, we use the Wordnet data to create a False Friends’ dataset for eleven language pairs. We also evaluate the efficacy of our dataset using previously available baseline cognate detection approaches. We also perform a manual evaluation with the help of lexicographers and release the curated gold-standard dataset with this paper. Keywords: cognate sets, Indian languages, cognate dataset, true cognates, false friends, gold data 1. Introduction and Motivation Cognates are words that have a common etymological origin. For e.g., the French and English word pair, Libert´e - Liberty, reveals itself to be a cognate through orthographic similarity. Automatic Cognate Detection (ACD) is the task of detecting such etymologically related words or word sets among different languages1. They share a formal and/or semantic affinity. Cognate words can facilitate the Second Language Acquisition (SLA) process, particularly between related languages. They have similar meanings and, therefore, can support the acquisition and/or learning of a non-native language. However, although they can accelerate vocabulary acquisition, learners also have to be aware of false friends and partial cognates. False friends are similar words that have distinct, unrelated meanings. For example, “gift” in German means “poison” unlike its English meaning. We illustrate the occurrence of one such example each for cognates and false friends’ for a pair of Indian languages in Table 1. In other cases, there are partial cognates i.e., similar words that have a common meaning only in some contexts. For example, the word “police” in French can translate to “police”, “policy” or “font”, depending on the context. Dictionaries often include information about cognates and false friends, and there are dictionaries (Hammer and Monod, 1976; Prado, 1993) exclusively devoted to them. In this paper, we describe the creation of three cognate datasets. First, we describe the digitization of one such Cognate dictionary named, “Tatsama Shabda Kosha” and its annotation with linked Wordnet IDs. With the help of a lexicographer, we perform the digitization of this dictio- nary. Further, …

Content truncated to 500 words