A Probabilistic Generative Model of Linguistic Typology

Proceedings of NAACL-HLT 2019, pages 1529–1540 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 1529 A Probabilistic Generative Model of Linguistic Typology Johannes Bjerva@ Yova Kementchedjhieva@ Ryan CotterellP,H Isabelle Augenstein@ @Department of Computer Science, University of Copenhagen PDepartment of Computer Science, Johns Hopkins University HDepartment of Computer Science and Technology, University of Cambridge bjerva,yova,augenstein@di.ku.dk, rdc42@cam.ac.uk Abstract In the principles-and-parameters framework, the structural features of languages depend on parameters that may be toggled on or off, with a single parameter often dictating the status of multiple features. The implied co- variance between features inspires our prob- abilisation of this line of linguistic inquiry— we develop a generative model of language based on exponential-family matrix factorisa- tion. By modelling all languages and fea- tures within the same architecture, we show how structural similarities between languages can be exploited to predict typological features with near-perfect accuracy, outperforming sev- eral baselines on the task of predicting held- out features. Furthermore, we show that lan- guage embeddings pre-trained on monolingual text allow for generalisation to unobserved lan- guages. This finding has clear practical and also theoretical implications: the results con- firm what linguists have hypothesised, i.e. that there are significant correlations between typo- logical features and languages. 1 Introduction Linguistic typologists dissect and analyse lan- guages in terms of their structural properties (Croft, 2002). For instance, consider the phonological property of word-final obstruent decoding: Ger- man devoices word-final obstruents (Zug is pro- nounced /zuk/), whereas English does not (dog is pronounced /d6g/). In the tradition of generative linguistics, one line of typological analysis is the principles-and-parameters framework (Chomsky, 1981), which posits the existence of a set of univer- sal parameters, switches as it were, that languages toggle. One arrives at a kind of factorial typol- ogy, to borrow terminology from optimality theory (Prince and Smolensky, 2008), through different settings of the parameters. Within the principle- and-parameters research program, then, the goal is Figure 1: Correlations between selected typological parameters. Feature values are classified according to head-directionality (head-initial +, head-final -, no dominant order o). For instance, ++ under Affixation means strongly suffixing. to identify the parameters that serve as axes, along which languages may vary. It is not enough, however, to simply write down the set of parameters available to language. In- deed, one of the most interesting facets of typol- ogy is that different parameters are correlated. To illustrate this point, we show a heatmap in Fig- ure 1 that shows the correlation between the values of selected parameters taken from a typological knowledge base (KB). Notice how head-final word order, for example, highly correlates with strong suffixation. The primary contribution of this work is a probabilisation of typology inspired by the principles-and-parameters framework. We assume a given set of typological parameters and develop a generative model of a language’s parameters, cast- ing the problem as a form of exponential-family ma- trix factorisation. We observe a binary matrix that encodes the settings of each parameter for each lan- guage. …

Content truncated to 500 words