A Universal Framework for Inductive Transfer Parsing across Multi-typed Treebanks

Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 12–22, Osaka, Japan, December 11-17 2016. A Universal Framework for Inductive Transfer Parsing across Multi-typed Treebanks Jiang Guo�, Wanxiang Che�, Haifeng Wang♩ and Ting Liu� �Center for Social Computing and Information Retrieval, Harbin Institute of Technology, China ♩Baidu Inc., China {jguo, car, tliu}@ir.hit.edu.cn wanghaifeng@baidu.com Abstract Various treebanks have been released for dependency parsing. Despite that treebanks may be- long to different languages or have different annotation schemes, they contain common syntactic knowledge that is potential to benefit each other. This paper presents a universal framework for transfer parsing across multi-typed treebanks with deep multi-task learning. We consider two kinds of treebanks as source: the multilingual universal treebanks and the monolingual hetero- geneous treebanks. Knowledge across the source and target treebanks are effectively transferred through multi-level parameter sharing. Experiments on several benchmark datasets in various languages demonstrate that our approach can make effective use of arbitrary source treebanks to improve target parsing models. 1 Introduction As a long-standing central problem in natural language processing (NLP), dependency parsing has been dominated by data-driven approaches for decades. The foundation of data-driven parsing is the avail- ability and scale of annotated training data (i.e., treebanks). Numerous efforts have been made towards the construction of treebanks which established the benchmark research on dependency parsing, such as the widely-used Penn Treebank (Marcus et al., 1993). However, the heavy cost of treebanking typically limits the existing treebanks in both scale and coverage of languages. To address the problem, a variety of authors have proposed to exploit existing heterogeneous tree- banks with different annotation schemes via grammar conversion (Niu et al., 2009), quasi-synchronous grammar features (Li et al., 2012) or shared feature representations (Johansson, 2013) for the enhance- ment of parsing models. Despite their effectiveness in specific datasets, these methods typically lack the scalability of exploiting richer source treebanks, such as cross-lingual treebanks. In this paper, we aim at developing a universal framework for transfer parsing that can exploit multi- typed source treebanks to improve parsing of a target treebank. Specifically, we will consider two kinds of source treebanks, that are multilingual universal treebanks and monolingual heterogeneous treebanks. Cross-lingual supervision has proven highly beneficial for parsing low-resource languages (Hwa et al., 2005; McDonald et al., 2011), implying that different languages have a great deal of common ground in grammars. But unfortunately, linguistic inconsistencies also exist in both typologies and lexical repre- sentations across languages. Figure 1(a) illustrates two sentences in German and English with universal dependency annotations. The typological differences (subject-verb-object order) results in the opposite directions of the dobj arcs, while the rest arcs remain consistent. Similar problems also come with monolingual heterogeneous treebanks. Figure 1(b) shows an English sentence annotated with respectively the universal dependencies which are content-head and the CONLL dependencies which instead take the functional heads. Despite the structural divergences, these treebanks express the syntax of the same language, thereby sharing a large amount of common knowledge that can be …

Content truncated to 500 words