On the Distribution of Deep Clausal Embeddings: A Large Cross-linguistic Study

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3938–3943 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 3938 On the Distribution of Deep Clausal Embeddings: A Large Cross-linguistic Study Dami´an E. Blasi1,2 Ryan Cotterell3 Lawrence Wolf-Sonkin4 Sabine Stoll1 Balthasar Bickel1 Marco Baroni5,6 1: University of Z¨urich 2: Max Planck Institute for the Science of Human History 3: Cambridge University 4: Johns Hopkins University 5: Facebook AI Research 6: Catalan Institution for Research and Advanced Studies Abstract Embedding a clause inside another (“the girl [who likes cars [that run fast]] has arrived”) is a fundamental resource that has been ar- gued to be a key driver of linguistic expres- siveness. As such, it plays a central role in fundamental debates on what makes human language unique, and how they might have evolved. Empirical evidence on the preva- lence and the limits of embeddings has how- ever been based on either laboratory setups or corpus data of relatively limited size. We in- troduce here a collection of large, dependency- parsed written corpora in 17 languages, that allow us, for the first time, to capture clausal embedding through dependency graphs and as- sess their distribution. Our results indicate that there is no evidence for hard constraints on embedding depth: the tail of depth distri- butions is heavy. Moreover, although deeply embedded clauses tend to be shorter, suggest- ing processing load issues, complex sentences with many embeddings do not display a bias to- wards less deep embeddings. Taken together, the results suggest that deep embeddings are not disfavored in written language. More gen- erally, our study illustrates how resources and methods from latest-generation big-data NLP can provide new perspectives on fundamental questions in theoretical linguistics. 1 Introduction In a prominent intellectual tradition, the infinitude of human expressivity (Humboldt, 1836) is rooted in a machinery that allows syntactic embedding at arbitrary depth (Chomsky, 1957, 1995). Re- gardless of the controversy around this proposal in terms of computational theory (Pullum and Scholz, 2010; Watamull et al., 2014), it remains an open issue to what extent languages in fact deploy struc- tures with arbitrarily deep embedding. Many lan- guages contain specific constructions that cap em- bedding depth at phrasal levels to one (e.g. unlike the girl who likes cars that run fast has arrived DET NSUBJ NSUBJ ACL:RELCL DOBJ NSUBJ ACL:RELCL ADVMOD AUX ROOT Figure 1: Example UD parse for sentence with maxi- mum embedding d = 2. English, Modern Greek compounds don’t allow re- cursive embedding; Ralli, 2013), although more radical constraints (Mithun, 1984; Everett, 2005; Evans and Levinson, 2009) seem to be rare and are avoided when languages evolve over time (Wid- mer et al., 2017). In terms of sentence production, embedding depths seem to be capped at moderate levels, likely because deeper embeddings place in- creasing demands on the brain’s processing system (Gildea and Temperley, 2010). Corpus studies of English, Pirah˜a, and a few other—mostly European—languages proposed constraints at one (Reich, 1969; Futrell et …

Content truncated to 500 words