Working Hard or Hardly Working: Challenges of Integrating Typology into Neural Dependency Parsers

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 5714–5720, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 5714 Working Hard or Hardly Working: Challenges of Integrating Typology into Neural Dependency Parsers Adam Fisch∗ Jiang Guo∗ Regina Barzilay Computer Science and Artificial Intelligence Laboratory Massachusetts Institute of Technology {fisch,jiang guo,regina}@csail.mit.edu Abstract This paper explores the task of leveraging typology in the context of cross-lingual de- pendency parsing. While this linguistic in- formation has shown great promise in pre- neural parsing, results for neural architec- tures have been mixed. The aim of our in- vestigation is to better understand this state- of-the-art. Our main findings are as fol- lows: 1) The benefit of typological informa- tion is derived from coarsely grouping lan- guages into syntactically-homogeneous clus- ters rather than from learning to leverage vari- ations along individual typological dimensions in a compositional manner; 2) Typology con- sistent with the actual corpus statistics yields better transfer performance; 3) Typological similarity is only a rough proxy of cross- lingual transferability with respect to parsing.1 1 Introduction Over the last decade, dependency parsers for resource-rich languages have steadily continued to improve. In parallel, significant research efforts have been dedicated towards advancing cross- lingual parsing. This direction seeks to capitalize on existing annotations in resource-rich languages by transferring them to the rest of the world’s over 7,000 languages (Bender, 2011). The NLP com- munity has devoted substantial resources towards this goal, such as the creation of universal annota- tion schemas, and the expansion of existing tree- banks to diverse language families. Nevertheless, cross-lingual transfer gains remain modest when put in perspective: the performance of transfer models can often be exceeded using only a handful of annotated sentences in the target language (Sec- tion 5). The considerable divergence of language structures proves challenging for current models. ∗The first two authors contributed equally. 1Code: github.com/ajfisch/TypologyParser One promising direction for handling these di- vergences is linguistic typology. Linguistic ty- pology classifies languages according to their structural and functional features. By explic- itly highlighting specific similarities and differ- ences in languages’ syntactic structures, typology holds great potential for facilitating cross-lingual transfer (O’Horan et al., 2016). Indeed, non- neural parsing approaches have already demon- strated empirical benefits of typology-aware mod- els (Naseem et al., 2012; T¨ackstr¨om et al., 2013; Zhang and Barzilay, 2015) While adding discrete typological attributes is straightforward for tradi- tional feature-based approaches, for modern neu- ral parsers finding an effective implementation choice is more of an open question. Not surpris- ingly, the reported results have been mixed. For instance, Ammar et al. (2016) found no benefit to using typology for parsing when using a neural- based model, while Wang and Eisner (2018) and Scholivet et al. (2019) did in several cases. There are many possible hypotheses that can at- tempt to explain the state-of-the-art. Might neu- ral models already implicitly learn typological in- formation on their own? Is the hand-specified …

Content truncated to 500 words