The Galactic Dependencies Treebanks: Getting More Data by Synthesizing New Languages

The Galactic Dependencies Treebanks: Getting More Data by Synthesizing New Languages Dingquan Wang and Jason Eisner Department of Computer Science, Johns Hopkins University {wdd,eisner}@jhu.edu Abstract We release Galactic Dependencies 1.0—a large set of synthetic languages not found on Earth, but annotated in Universal Dependen- cies format. This new resource aims to pro- vide training and development data for NLP methods that aim to adapt to unfamiliar lan- guages. Each synthetic treebank is produced from a real treebank by stochastically permut- ing the dependents of nouns and/or verbs to match the word order of other real languages. We discuss the usefulness, realism, parsabil- ity, perplexity, and diversity of the synthetic languages. As a simple demonstration of the use of Galactic Dependencies, we consider single-source transfer, which attempts to parse a real target language using a parser trained on a “nearby” source language. We find that including synthetic source languages some- what increases the diversity of the source pool, which significantly improves results for most target languages. 1 Motivation Some potential NLP tasks have very sparse data by machine learning standards, as each of the IID train- ing examples is an entire language. For instance: • typological classification of a language on var- ious dimensions; • adaptation of any existing NLP system to new, low-resource languages; • induction of a syntactic grammar from text; • discovery of a morphological lexicon from text; • other types of unsupervised discovery of lin- guistic structure. Given a corpus or other data about a language, we might aim to predict whether it is an SVO lan- guage, or to learn to pick out its noun phrases. For such problems, a single training or test example cor- responds to an entire human language. Unfortunately, we usually have only from 1 to 40 languages to work with. In contrast, machine learn- ing methods thrive on data, and recent AI successes have mainly been on tasks where one can train richly parameterized predictors on a huge set of IID (input, output) examples. Even 7,000 training examples— one for each language or dialect on Earth—would be a small dataset by contemporary standards. As a result, it is challenging to develop systems that will discover structure in new languages in the same way that an image segmentation method, for example, will discover structure in new images. The limited resources even make it challenging to de- velop methods that handle new languages by un- supervised, semi-supervised, or transfer learning. Some such projects evaluate their methods on new sentences of the same languages that were used to develop the methods in the first place—which leaves one worried that the methods may be inadvertently tuned to the development languages and may not be able to discover correct structure in other languages. Other projects take care to hold out languages for evaluation (Spitkovsky, 2013; Cotterell et al., 2015), but then are left with only a few development lan- guages on which to experiment with different unsu- pervised methods and their hyperparameters. If we had many …

Content truncated to 500 words