A Natural Language Processing Infrastructure for Turkish

A Natural Language Processing Infrastructure for Turkish A. C. Cem SAY Department of Computer Engineering, Bogaziçi University, Bebek,�stanbul say@boun.edu.tr Özlem ÇET�NO�LU Faculty of Engineering and Natural Sciences, Sabancı University, Tuzla,�stanbul ozlemc@sabanciuniv.edu �eniz DEM�R Department of Computer Engineering, Bogaziçi University Bebek,�stanbul sdemir@cse.yeditepe.edu.tr Fatih Ö�ÜN Department of Computer Engineering, Bogaziçi University Bebek,�stanbul fatih_ogun@yahoo.com Abstract We built an open-source software platform in- tended to serve as a common infrastructure that can be of use in the development of new applica- tions involving the processing of Turkish. The platform incorporates a lexicon, a morphological analyzer/generator, and a DCG parser/generator that translates Turkish sentences to predicate logic formulas, and a knowledge base frame- work. Several developers have already utilized the platform for a variety of applications, includ- ing conversation programs and an artificial per- sonal assistant, tools for automatic analysis of rhyme and meter in Turkish folk poems, a proto- type sentence-level translator between Albanian, Turkish, and English, natural language interfaces for generating SQL queries and JAVA code, as well as a text tagger used for collecting statistics about Turkish morpheme order for a speech rec- ognition algorithm. The results indicate the adaptability of the infrastructure to different kinds of applications and how it facilitates im- provements and modifications. Introduction The obvious potential of natural language processing technology for economic, social and cultural pro- gress can be realized more comprehensively if NLP techniques applicable to a wider selection of the lan- guages of the world are developed. Before the full- scale treatment of a new language can start, a con- siderable amount of effort has to be invested to computerize the lexical, morphological and syntactic specifics of that language, which would be required by any nontrivial application. We built an open-source software platform in- tended to serve as a common infrastructure that can be of use in the development of new applications involving the processing of Turkish. The platform, named TOY (Çetino�lu 2001), is essentially a big set of predicates in the logic programming language Prolog. The choice of Prolog, which was designed specifically with computational linguistics applica- tions in mind, as the implementation language for our software has natural consequences for the knowledge representation setup to be used by other programs built on our platform. Prolog is based on first-order predicate calculus, it allows knowledge items to be represented in terms of logic-style facts and rules, and a built-in theorem prover drives the execution of Prolog queries. The TOY program’s internal organization into source files reflects the three different levels (see Figure 1) on which text-based NLP applications can be based. In terms of that figure, processing at a “deeper” level necessitates all components of “shal- lower” levels. In this paper, we describe this infrastructure and how it was adapted to a variety of applications. Sec- tion 2 gives a brief overview of the infrastructure. Section 3 presents the applications based on it. 1 Infrastructure The TOY platform is formed of a lexicon that con- tains most of the Turkish morphemes (either root or …

Content truncated to 500 words