magyarlanc: A Tool for Morphological and Dependency Parsing of Hungarian

Proceedings of Recent Advances in Natural Language Processing, pages 763–771, Hissar, Bulgaria, 7-13 September 2013. magyarlanc: A Toolkit for Morphological and Dependency Parsing of Hungarian J´anos Zsibrita1, Veronika Vincze1,2 and Rich´ard Farkas1 1Department of Informatics, University of Szeged {zsibrita,rfarkas}@inf.u-szeged.hu 2Hungarian Academy of Sciences Research Group on Artificial Intelligence vinczev@inf.u-szeged.hu Abstract Hungarian is the stereotype of morpholog- ically rich and free word order languages. Here, we introduce magyarlanc, a nat- ural language toolkit developed for the linguistic preprocessing – segmentation, morphological analysis, POS-tagging and dependency parsing – of Hungarian texts. We hope that the free availability of the toolkit fosters the research not just on the Hungarian language but on all the mor- phologically rich languages in general. The main novelties of the tool are the ap- plication of a new harmonized morpholog- ical coding system of Hungarian, the data- driven approach and the integration of a dependency parser. The system is imple- mented in JAVA, hence it can be used in a platform-independent way. 1 Introduction For end user natural language processing applica- tions, it is essential to have access to a basic lin- guistic analyzer tool on the target language, in or- der to prevent reinventing the wheel every time. In this paper, we present magyarlanc, a basic lin- guistic analyzer toolkit developed for Hungarian. Hungarian is a morphologically rich language with free word order (i.e. leaving aside the issue of the internal structure of NPs, most sentence-level syntactic information in Hungarian is conveyed by morphology, not by configuration). A large part of the methodology for morphosyntactic anal- ysis has been developed for English. However, the linguistic analysis of morphologically rich and free word order languages requires special tech- niques. Hence, it was not sufficient to simply em- ploy available tools and retrain on Hungarian cor- pora, we had to modify/adapt them. We hope that our findings and experiences gained during this adaptation process are useful for everybody deal- ing with morphologically rich – especially agglu- tinative – languages. magyarlanc is enriched with a sentence split- ter and tokenizer, a morphological analyzer, a POS-tagger and a dependency parser, each of them fine-tuned for the characteristics of Hungarian. The main novelties of magyarlanc are the fol- lowing (each of the three criteria is unique among Hungarian-oriented linguistic analyzers): • It is data-driven. Every module was system- atically trained and evaluated on the Szeged Corpus and Szeged Dependency Treebank (82K sentences with manual annotation). • It is an integrated toolkit, starting from raw text outputs to dependency parses. • It is implemented fully in JAVA (incorpora- tion to big systems is straightforward). magyarlanc is freely available for research purposes at http://www.inf.u-szeged. hu/rgai/magyarlanc. The structure of the paper is the following. First, we provide a summary of the grammati- cal features of Hungarian, which is followed and a short description of Hungarian morphological coding systems. Then we present the modules of magyarlanc. We also test the efficiency of magyarlanc and provide results on morpholog- ical and dependency parsing. 2 Grammatical Features of …

Content truncated to 500 words