Uncovering Probabilistic Implications in Typological Knowledge Bases

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3924–3930 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 3924 Uncovering Probabilistic Implications in Typological Knowledge Bases Johannes Bjerva@ Yova Kementchedjhieva@ Ryan CotterellP,H Isabelle Augenstein@ @Department of Computer Science, University of Copenhagen PDepartment of Computer Science, Johns Hopkins University HDepartment of Computer Science and Technology, University of Cambridge bjerva,yova,augenstein@di.ku.dk, rdc42@cam.ac.uk Abstract The study of linguistic typology is rooted in the implications we find between linguistic features, such as the fact that languages with object-verb word ordering tend to have post- positions. Uncovering such implications typi- cally amounts to time-consuming manual pro- cessing by trained and experienced linguists, which potentially leaves key linguistic univer- sals unexplored. In this paper, we present a computational model which successfully iden- tifies known universals, including Greenberg universals, but also uncovers new ones, wor- thy of further linguistic investigation. Our ap- proach outperforms baselines previously used for this problem, as well as a strong baseline from knowledge base population. 1 Introduction Linguistic typology is concerned with mapping out the relationships between languages with reference to structural and functional properties (Croft, 2002). A typologist may ask, for instance, how a language encodes syntactic features and relationships. Does it place its verbs before objects or after, and does it have prepositions or postpositions? It is well estab- lished that many features of languages are highly correlated, sometimes to the extent that they imply each other. Based on this observation, Greenberg (1963) establishes the notion of implicational uni- versals, i.e., cases where the presence of one feature strictly implies the presence of another. Universals are important to investigate as they of- fer insight into the inner workings of language and define the space of plausible languages. Universals can aid cognitive scientists examining the underly- ing processes of language, as there arguably is a cognitive reason for why, e.g., languages with OV ordering are postpositional (Greenberg, 1963). In the context of natural language processing (NLP), when creating synthetic data for multilingual NLP, SV OV SVO Noun- Adjective Figure 1: Visualisation of a section of our induced graphical model. Observing the features in the left- most nodes (SV, OV, and Noun-Adjective), can we cor- rectly infer the value of the right-most node (SVO)? one should consider universals to maintain the plau- sibility of the data (Wang and Eisner, 2016). Com- putational typology can furthermore be used to in- duce language representations, useful in, e.g., lan- guage modelling ( ¨Ostling and Tiedemann, 2017) and syntactic parsing (de Lhoneux et al., 2018). In this paper, we argue that the deterministic Greenbergian view of implications (Greenberg, 1963) is outdated. Instead, we suggest that a proba- bilistic view of implications is more suitable, and define the notion of a probabilistic typological im- plication as a certain conditional probability dis- tribution. We do this by first placing a joint distri- bution over the vector of typological features, and then marginalising out all features other than the two under …

Content truncated to 500 words