SHR++: An Interface for Morpho-syntactic Annotation of Sanskrit Corpora

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 7069–7076 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 7069 SHR++: An Interface for Morpho-syntactic annotation of Sanskrit Corpora Amrith Krishna#, Shiv Vidhyut!, Dilpreet ChawlaG, Sruti Sambhavip, Pawan Goyal: #IT University of Copenhagen, !IIITDM Kancheepuram, GIIIT Kalyani pNIT Rourkela, :IIT Kharagpur amrith@iitkgp.ac.in, pawang@cse.iitkgp.ac.in Abstract We propose a web-based annotation framework, SHR++, for morpho-syntactic annotation of corpora in Sanskrit. SHR++ is designed to generate annotations for the word-segmentation, morphological parsing and dependency analysis tasks in Sanskrit. It incorporates analyses and predictions from various tools designed for processing texts in Sanskrit, and utilises them to ease the cognitive load of the human annotators. Specifically, SHR++ uses Sanskrit Heritage Reader (Goyal and Huet, 2016), a lexicon driven shallow parser for enumerating all the phonetically and lexically valid word splits along with their morphological analyses for a given string. This would help the annotators in choosing the solutions, rather than performing the segmentations by themselves. Further, predictions from a word segmentation tool (Krishna et al., 2018) are added as suggestions that can aid the human annotators in their decision making. Our evaluation shows that enabling this segmentation suggestion component reduces the annotation time by 20.15 %. SHR++ can be accessed online at http://vidhyut97.pythonanywhere.com/ and the codebase, for the independent deployment of the system elsewhere, is hosted at https://github.com/iamdsc/smart-sanskrit-annotator. Keywords: low-resource languages, Sanskrit, annotation, dependency trees, morphological analysis, word segmentation, visu- alization, GUI, crowd-sourcing, treebanking 1. Introduction Sanskrit is a classical language (Coulson, 1992; Sapir, 2004), and used to be the ‘lingua franca’ for the scientific, literary and philosophical discourse in the Indian subconti- nent for more than 3 millennia. The language has over 30 million extant manuscripts potent for digitisation, one hun- dred times those in Greek and Latin combined (Goyal et al., 2012). However, a major source of concern for the process- ing of texts in a low-resource language like Sanskrit is the lack of availability of labelled data. Acquiring task-specific annotations can be linguistically involved and time consum- ing, making it a challenge in itself. In this work, we present ‘SHR++’, a web-based annotation framework that would assist a professional annotator1 to perform annotations for three different tasks, ranging from word-segmentation, mor- phological parsing and dependency parsing. Word-segmentation is a challenging task in Sanskrit, as the writings in Sanskrit follow a phonemic orthography. Here the words in the sentence often undergo phonetic trans- formations at the juncture of the word boundaries, which not only obscure the word boundaries but also modify the graphemes present in the string. This euphonic assimi- lation of phones is formalised as Sandhi. Sandhi, while mostly deterministic in generation, is strongly ambiguous in analysis. This makes segmentation more difficult than languages such as Chinese, where the words are combined without any euphonic assimilation at the boundary (Goyal and Huet, 2016). In this work, we extend the function- alities of a lexicon-driven shallow parser and annotation framework for Sanskrit, Sanskrit Heritage …

Content truncated to 500 words