Multitask Easy-First Dependency Parsing: Exploiting Complementarities of Different Dependency Representations

Proceedings of the 28th International Conference on Computational Linguistics, pages 2497–2508 Barcelona, Spain (Online), December 8-13, 2020 2497 Multitask Easy-First Dependency Parsing: Exploiting Complementarities of Different Dependency Representations Yash Kankanampati, Joseph Le Roux,† Nadi Tomeh,† Dima Taji,‡ Nizar Habash‡ Indian Institute of Technology - Dhanbad Université Sorbonne Paris Nord,† New York University Abu Dhabi‡ yash.15je001733@am.iitism.ac.in {leroux,tomeh}@lipn.fr, {dima.taji,nizar.habash}@nyu.edu Abstract We present a parsing model for projective dependency trees which takes advantage of the exis- tence of complementary dependency annotations for a language. This is the case for Arabic with the availability of CATiB and UD treebanks. Our system performs syntactic parsing according to both annotation types jointly as a sequence of arc-creating operations following the Easy-First approach, and partially created trees for one annotation type are also available to the other as features for the score function. This method gives error reduction of 9.9% on CATiB and 6.1% on UD compared to a single-task baseline, and ablation tests show that the main contribution of this reduction is given by sharing tree representation between tasks, and not simply sharing BiLSTM layers as is usually performed in NLP multitask systems. 1 Introduction Dependency parsing is the task of assigning a syntactic structure to a sentence by linking its words with binary asymmetrical typed relations. In addition to syntactic information, dependency representations encode some semantic aspects of the sentence which make them important to downstream applications including sentiment analysis (Tai et al., 2015) and information extraction (Miwa and Bansal, 2016). In this paper we are interested in Arabic dependency parsing for which two formalisms have been developed (See §2). The first is the Columbia Arabic Treebank (CATiB) representation (Habash and Roth, 2009), which is inspired by Arabic traditional grammar and which focus on modeling syntactic and morpho-syntactic agreement and case assignment. The second is the Universal Dependency (UD) representation (Taji et al., 2017), which has relatively more focus on semantic/thematic relations, and which is coordinated in design with a number of other languages (Nivre et al., 2017). While previous work on Arabic dependency parsing (Marton et al., 2013; Taji et al., 2017) tackled these formalisms separately, we argue that they stand to benefit from multitask learning (MTL) (Caruana, 1993). MTL allows for more training data to be exploited while benefiting from the structural or statistical similarities between the tasks. We therefore propose to learn CATiB and UD dependency trees jointly on the same input sentences using parallel treebanks. Deep neural networks are particularly suited for multitask scenarios via straightforward parameter and representation sharing. Some hidden layers can be shared across all tasks while output layers are kept separate. In fact, most deep learning architectures for language processing start with sequential encoding components such as BiLSTMs or Transformer layers which can be readily shared across multiple tasks. This approach is widely applicable even to tasks that use very different formalisms and do not have par- allel annotations (Søgaard and Goldberg, 2016; Hashimoto et al., 2017). This type of sharing has also been shown to benefit …

Content truncated to 500 words