A Distributed Platform for Sanskrit Processing

Proceedings of COLING 2012: Technical Papers, pages 1011–1028, COLING 2012, Mumbai, December 2012. A Distributed Platform for Sanskrit Processing Pawan Goyal1 Gérard Huet1 Amba Kulkarni2 Peter Scharf3 Ralph Bunker4 (1) Inria Paris-Rocquencourt (2) Department of Sanskrit Studies, University of Hyderabad (3) Blaise Pascal Chair, Université Paris 7, and The Sanskrit Library (4) Maharishi University of Management, Fairfield, Iowa Pawan.Goyal@inria.fr, Gerard.Huet@inria.fr, apksh@uohyd.ernet.in, Peter.Scharf@inria.fr, rbunker@lisco.com Abstract Sanskrit, the classical language of India, presents specific challenges for computational linguistics: exact phonetic transcription in writing that obscures word boundaries, rich morphology and an enormous corpus, among others. Recent international cooperation has developed innovative solutions to these problems and significant resources for linguistic research. Solutions include efficient segmenting and tagging algorithms and dependency parsers based on constraint programming. The integration of lexical resources, text archives and linguistic software is achieved by distributed interoperable Web services. Resources include a morphological tagger and tagged corpus. Keywords: Indian language technology, Resources and annotation, Morphology and POS tagging, Parsing. 1011 1 Introduction Formal and computational linguistics was dominated by English at its inception and developed in subsequent decades primarily in the environment of European languages. More recently there has been a concerted effort to undertake formal linguistic analysis of a wide variety of languages, with particular interest in those with dramatically different features, and to enrich linguistic theory to account for linguistic variety. In spite of this effort, analytic structures and procedures utilized in formal linguistics remain dominated by those invented for, and most suitable for, English and other European languages. Linguistic theory remains unduly weighted in favor of European languages even as their extension to the variety of the world’s languages involves undue complication thereby revealing their inadequacy in representing language universally. The inadequacy of contemporary computational methods is vividly apparent in the analysis of Sanskrit. Recent worldwide collab- oration to overcome the challenges to conducting computational linguistic research on Sanskrit offers insights into methods and procedures that may be useful generally for languages that differ markedly from western European languages. 1.1 Sanskrit Sanskrit is the primary culture-bearing language of India, with a continuous production of literature in all fields of human endeavor over the course of four millennia. Preceded by a strong oral tradition of knowledge transmission, records of written Sanskrit remain in the form of inscriptions dating back to the first century B.C.E. Extant manuscripts in Sanskrit number over 30 million - one hundred times those in Greek and Latin combined - constituting the largest cultural heritage that any civilization has produced prior to the invention of the printing press. Sanskrit works include extensive epics, subtle and intricate philosophical, mathematical, and scientific treatises, and imaginative and rich literary, poetic, and dramatic texts. The primary language of the Vedic civilization, Sanskrit developed constrained by a strong grammatical tradition stemming from the fairly complete grammar composed by P¯an.ini by the fourth century B.C.E. In addition to serving as an object of study in academic institutions, the Sanskrit language persists in the recitation of hymns in daily worship and …

Content truncated to 500 words