EstNLTK - NLP Toolkit for Estonian

ESTNLTK - NLP Toolkit for Estonian Siim Orasmaa, Timo Petmanson, Alexander Tkachenko, Sven Laur, Heiki-Jaan Kaalep Institute of Computer Science, University of Tartu Liivi 2, 50409 Tartu, Estonia siim.orasmaa@ut.ee,tpetmanson@gmail.com,aleksandr.tkatsenko@ut.ee,swen@ut.ee,heiki-jaan.kaalep@ut.ee Abstract Although there are many tools for natural language processing tasks in Estonian, these tools are very loosely interoperable, and it is not easy to build practical applications on top of them. In this paper, we introduce a new Python library for natural language processing in Estonian, which provides a unified programming interface for various NLP components. The ESTNLTK toolkit provides utilities for basic NLP tasks including tokenization, morphological analysis, lemmatisation and named entity recognition as well as offers more advanced features such as a clause segmentation, temporal expression extraction and normalization, verb chain detection, Estonian Wordnet integration and rule-based information extraction. Accompanied by a detailed API documentation and comprehensive tutorials, ESTNLTK is suitable for a wide range of audience. We believe ESTNLTK is mature enough to be used for developing NLP-backed systems both in industry and research. ESTNLTK is freely available under the GNU GPL version 2+ license, which is standard for academic software. Keywords: natural language processing, Python, Estonian language 1. Introduction Estonian scientific community has recently enjoyed an ac- tive period, when a number of major NLP components have been developed under free open source licenses. Unfortu- nately, these tools are very loosely interoperable. They use different programming languages, data formats and have specific hardware and software requirements. Such sit- uation complicates their usage in both software develop- ment and educational setting. In this paper, we introduce ESTNLTK, a Python library for natural language process- ing in Estonian, which addresses this issue. The ESTNLTK toolkit glues together existing software components and makes them easily accessible via a uni- fied programming interface. It provides utilities for basic NLP tasks including tokenization, morphological analysis, lemmatisation and named entity recognition as well as of- fers more advanced features such as a clause segmentation, temporal expression extraction, verb chain detection, Esto- nian Wordnet integration and grammar-based information extraction. The ESTNLTK toolkit is written in Python programming language and borrows design ideas from popular NLP toolkits TextBlob and NLTK. Users familiar with these tools can easily get started with ESTNLTK. Accompanied by a detailed API documentation and comprehensive tutori- als, ESTNLTK is suitable for a wide range of audience. We believe ESTNLTK is mature enough to be used for devel- oping NLP-backed systems both in industry and research. Additionally, ESTNLTK is a good environment for teach- ing NLP for students. Although ESTNLTK is explicitly targeted for the Estonian language, the architecture is quite generic and the toolkit can be used as a template for building analogous toolkits for other languages. The only limiting factor is the availability of external NLP components that must be replaced. The ESTNLTK toolkit is available under the GNU GPL version 2+ license from https://github.com/ estnltk/estnltk. The library works on Linux, Win- dows and Mac OS X and supports Python versions 2.7 and 3.4. 2. Related Work There is …

Content truncated to 500 words