Semi-supervised learning of morphological paradigms and lexicons

Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics, pages 569–578, Gothenburg, Sweden, April 26-30 2014. c⃝2014 Association for Computational Linguistics Semi-supervised learning of morphological paradigms and lexicons Malin Ahlberg Spr˚akbanken University of Gothenburg malin.ahlberg@gu.se Markus Forsberg Spr˚akbanken University of Gothenburg markus.forsberg@gu.se Mans Hulden University of Helsinki mans.hulden@helsinki.fi Abstract We present a semi-supervised approach to the problem of paradigm induction from inflection tables. Our system ex- tracts generalizations from inflection ta- bles, representing the resulting paradigms in an abstract form. The process is in- tended to be language-independent, and to provide human-readable generalizations of paradigms. The tools we provide can be used by linguists for the rapid cre- ation of lexical resources. We evaluate the system through an inflection table recon- struction task using Wiktionary data for German, Spanish, and Finnish. With no additional corpus information available, the evaluation yields per word form ac- curacy scores on inflecting unseen base forms in different languages ranging from 87.81% (German nouns) to 99.52% (Span- ish verbs); with additional unlabeled text corpora available for training the scores range from 91.81% (German nouns) to 99.58% (Spanish verbs). We separately evaluate the system in a simulated task of Swedish lexicon creation, and show that on the basis of a small number of inflection tables, the system can accurately collect from a list of noun forms a lexicon with in- flection information ranging from 100.0% correct (collect 100 words), to 96.4% cor- rect (collect 1000 words). 1 Introduction Large scale morphologically accurate lexicon con- struction for natural language is a very time- consuming task, if done manually. Usually, the construction of large-scale lexical resources pre- supposes a linguist who constructs a detailed mor- phological grammar that models inflection, com- pounding, and other morphological and phonolog- ical phenomena, and additionally performs a man- ual classification of lemmas in the language ac- cording to their paradigmatic behavior. In this paper we address the problem of lexicon construction by constructing a semi-supervised system that accepts concrete inflection tables as in- put, generalizes inflection paradigms from the ta- bles provided, and subsequently allows the use of unannotated corpora to expand the inflection ta- bles and the automatically generated paradigms.1 In contrast to many machine learning ap- proaches that address the problem of paradigm ex- traction, the current method is intended to produce human-readable output of its generalizations. That is, the paradigms provided by the system can be inspected for errors by a linguist, and if neces- sary, corrected and improved. Decisions made by the extraction algorithms are intended to be trans- parent, permitting morphological system develop- ment in tandem with linguist-provided knowledge. Some of the practical tasks tackled by the sys- tem include the following: • Given a small number of known inflection ta- bles, extract from a corpus a lexicon of those lemmas that behave like the examples pro- vided by the linguist. • Given a large number of inflection tables— such as those provided by the crowdsourced lexical resource, Wiktionary—generalize the tables …

Content truncated to 500 words