Fine-Grained Prediction of Syntactic Typology: Discovering Latent Structure with Supervised Learning

Fine-Grained Prediction of Syntactic Typology: Discovering Latent Structure with Supervised Learning Dingquan Wang and Jason Eisner Department of Computer Science, Johns Hopkins University {wdd,eisner}@jhu.edu Abstract We show how to predict the basic word-order facts of a novel language given only a cor- pus of part-of-speech (POS) sequences. We predict how often direct objects follow their verbs, how often adjectives follow their nouns, and in general the directionalities of all depen- dency relations. Such typological properties could be helpful in grammar induction. While such a problem is usually regarded as unsu- pervised learning, our innovation is to treat it as supervised learning, using a large collec- tion of realistic synthetic languages as train- ing data. The supervised learner must identify surface features of a language’s POS sequence (hand-engineered or neural features) that cor- relate with the language’s deeper structure (la- tent trees). In the experiment, we show: 1) Given a small set of real languages, it helps to add many synthetic languages to the training data. 2) Our system is robust even when the POS sequences include noise. 3) Our system on this task outperforms a grammar induction baseline by a large margin. 1 Introduction Descriptive linguists often characterize a human language by its typological properties. For instance, English is an SVO-type language because its basic clause order is Subject-Verb-Object (SVO), and also a prepositional-type language because its adpositions normally precede the noun. Identifying basic word order must happen early in the acqui- sition of syntax, and presumably guides the initial interpretation of sentences and the acquisition of a finer-grained grammar. In this paper, we propose a method for doing this. While we focus on word order, one could try similar methods for other typo- logical classifications (syntactic, morphological, or phonological). The problem is challenging because the lan- guage’s true word order statistics are computed from syntax trees, whereas our method has access only to a POS-tagged corpus. Based on these POS se- quences alone, we predict the directionality of each type of dependency relation. We define the direc- tionality to be a real number in [0, 1]: the fraction of tokens of this relation that are “right-directed,” in the sense that the child (modifier) falls to the right of its parent (head). For example, the dobj relation points from a verb to its direct object (if any), so a di- rectionality of 0.9—meaning that 90% of dobj de- pendencies are right-directed—indicates a dominant verb-object order. (See Table 1 for more such exam- ples.) Our system is trained to predict the relative frequency of rightward dependencies for each of 57 dependency types from the Universal Dependencies project (UD). We assume that all languages draw on the same set of POS tags and dependency relations that is proposed by the UD project (see §3), so that our predictor works across languages. Why do this? Liu (2010) has argued for using these directionality numbers in [0, 1] as fine-grained and robust typological descriptors. We believe that these directionalities could also be used to …

Content truncated to 500 words