Urdu Word Embeddings

Urdu Word Embeddings Samar Haider University of Engineering and Technology, Lahore Pakistan samar.haider@kics.edu.pk Abstract Representing words as vectors which encode their semantic properties is an important component in natural language processing. Recent advances in distributional semantics have led to the rise of neural network-based models that use unsupervised learning to represent words as dense, distributed vectors, called ‘word embeddings’. These embeddings have led to breakthroughs in performance in multiple natural language processing applications, and also hold the key to improving natural language processing for low-resource languages by helping machine learning algorithms learn patterns more easily from these richer representations of words, thereby allowing better generalization from less data. In this paper, we train the skip-gram model on more than 140 million Urdu words to create the first large-scale word embeddings for the Urdu language. We analyze the quality of the learned embeddings by looking at the closest neighbours to different words in the vector space and find that they capture a high degree of syntactic and semantic similarity between words. We evaluate this quantitatively by experimenting with different vector dimensionalities and context window sizes and measuring their performance on Urdu translations of standard word similarity tasks. The embeddings are made freely available in order to advance research on Urdu language processing. Keywords: Urdu, low-resource languages, vector representations, distributional semantics, word embeddings 1. Introduction Urdu is an Indo-Aryan language that is the national lan- guage and lingua franca of Pakistan, and an official lan- guage of multiple states of India. There are 109 million speakers of Urdu in Pakistan and 51 million speakers in In- dia. Urdu is also widely spoken across the rest of the world, with more than 163 million total speakers in all countries 1. Despite its widespread use, both in South Asia and among people of South Asian descent across the world, Urdu remains a low-resource language with few corpora and datasets of appreciable size available for computational tasks. This is not an uncommon phenomenon among low- resource languages, since creation of such resources re- quires significant time and manpower. However, without sufficient labelled data, it is very difficult to build natural language processing systems that can learn useful patterns which generalize well. This perennial lack of data leads to little research performed on these languages, which in turn leads to few resources created by research. One way to break out of this loop is to learn higher-level, complex representations of words and phrases that can then be used as input to bootstrap other natural language pro- cessing systems downstream. The area of distributional semantics focuses on creating just such representations, wherein semantically similar words are assigned similar representations. Recently, there has been much focus on using neural networks to learn vector representations of words by modelling the task as one of predicting surround- ing words from a target word. One advantage of such techniques is that they use unsupervised learning and don’t require annotated corpora, which are rare. They can in- stead be trained on larger, …

Content truncated to 500 words