Embeddings models for Buddhist Sanskrit

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 3861–3871 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 3861 Embeddings models for Buddhist Sanskrit Ligeia Lugli1, Matej Martinc2, Andraˇz Pelicon2, Senja Pollak2 1Mangalam Research Center for Buddhist Languages, Berkeley, CA, USA 2Joˇzef Stefan Institute, Ljubljana, Slovenia ligeia.lugli@kcl.ac.uk; {matej.martinc,andraz.pelicon,senja.pollak}@ijs.si Abstract The paper presents novel resources and experiments for Buddhist Sanskrit, broadly defined here as including all the varieties of Sanskrit in which Buddhist texts have been transmitted. We release a novel corpus of Buddhist texts, a novel corpus of general Sanskrit and word similarity and word analogy datasets for intrinsic evaluation of Buddhist Sanskrit embeddings models. We compare the performance of word2vec and fastText static embeddings models, with default and optimized parameter settings, as well as contextual models BERT and GPT-2, with different training regimes (including a transfer learning approach using the general Sanskrit corpus) and different embeddings construction regimes (given the encoder layers). The results show that for semantic similarity the fastText embeddings yield the best results, while for word analogy tasks BERT embeddings work the best. We also show that for contextual models the optimal layer combination for embedding construction is task dependent, and that pretraining the contextual embeddings models on a reference corpus of general Sanskrit is beneficial, which is a promising finding for future development of embeddings for less-resourced languages and domains. Keywords: Buddhist Sanskrit, embeddings, intrinsic evaluation 1. Introduction This study presents the evaluation of the performance of static and contextual embeddings models trained on a small historical corpus of Buddhist Sanskrit literature. This literature constitutes the textual foundation of Mah¯ay¯ana, one of the main branches of Buddhism, which flourished in India from around the first couple of centuries BCE to the XII century CE. Despite extensive scholarly endeavors, much uncertainty still surrounds this body of literature, especially regarding matters of chronology, authorship, and compositional history. Moreover, the lexicographic documentation of its vocabulary remains largely outdated (Lugli, 2018). While many world literatures and languages have benefited from the adoption of computational methods, the study of Buddhist Sanskrit sources still relies almost exclusively on traditional methods, such as close reading and philology. This is likely due to the scarcity of language resources available for Sanskrit in general and especially for the variety of Sanskrit used in Buddhist texts, which is characterized by domain-specific vocabulary, morphological patterns heavily influenced by local vernaculars and abundant spelling variation.1 We aim to improve on this situation by introducing novel corpora and language models to extend the resources available for both general and Buddhist Sanskrit. We also seek to contribute to the Digital Humanities debate over the feasibility and relative advantages of using static and contextual word embeddings with small domain- 1Note that in this study, we refer to the Sanskrit used in Buddhist literature as ’Buddhist Sanskrit’, regardless of the level of vernacular influence instantiated in each text. specific historical corpora (Wevers and Koolen, 2020), and more generally to the current developments …

Content truncated to 500 words