From SPMRL to NMRL: What Did We Learn (and Unlearn) in a Decade of Parsing Morphologically-Rich Languages (MRLs)?

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7396–7408 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 7396 From SPMRL to NMRL: What Did We Learn (and Unlearn) in a Decade of Parsing Morphologically-Rich Languages (MRLs)? Reut Tsarfaty Dan Bareket Stav Klein Bar Ilan University, Ramat-Gan, IsraelAmit Seker reut.tsarfaty@biu.ac.il {dbareket,klein.stav,aseker00}@gmail.com Abstract It has been exactly a decade since the first es- tablishment of SPMRL, a research initiative unifying multiple research efforts to address the peculiar challenges of Statistical Parsing for Morphologically-Rich Languages (MRLs). Here we reflect on parsing MRLs in that decade, highlight the solutions and lessons learned for the architectural, modeling and lex- ical challenges in the pre-neural era, and ar- gue that similar challenges re-emerge in neu- ral architectures for MRLs. We then aim to offer a climax, suggesting that incorporating symbolic ideas proposed in SPMRL terms into nowadays neural architectures has the poten- tial to push NLP for MRLs to a new level. We sketch a strategies for designing Neural Mod- els for MRLs (NMRL), and showcase prelim- inary support for these strategies via investi- gating the task of multi-tagging in Hebrew, a morphologically-rich, high-fusion, language. 1 Introduction The ability to process natural language data and to automatically extract structured meanings out of them has always been the hallmark of Artificial Intelligence (AI), and today it is also of immense practical value in downstream technological appli- cations for Information Extraction, Text Analytics, and diverse Data Science applications. The intro- duction of deep learning models (Goodfellow et al., 2016) into Natural Language Processing (NLP) has led to an explosion in the Neural models and pre- training techniques applied to NLP tasks — from classical tasks as tagging and parsing to end-to- end tasks as machine translation and question an- swering — raising the performance bar on these tasks to an all-times peak. So far though, these advances have been reported mostly for English. Can these advances carry over to languages that are typologically vastly different from English, such as Morphologically-Rich Languages? The term Morphologically-Rich Languages (MRLs) refers to languages such as Arabic, He- brew, Turkish or Maltese, in which significant in- formation is expressed morphologically, e.g., via word-level variation, rather than syntactically, e.g., via fixed word-order and periphrastic constructions, as in English. These properties lead to diverse and ambiguous structures, accompanied with huge lex- ica, which in turn make MRLs notoriously hard to parse (Nivre et al., 2007; Tsarfaty, 2013). A decade ago, Tsarfaty et al. (2010) put forth three overarch- ing challenges for the MRLs research community: (i) The Architectural Challenge: What input units are adequate for processing MRLs? (ii) The Modeling Challenge: What modeling assumptions are adequate for MRLs? (iii) The Lexical Challenge: How can we cope with extreme data sparseness in MRLs lexica? For NLP in the pre-neural era, effective solutions have been proposed and successfully applied to ad- dress each of these challenges for MRLs, using data from MRLs treebanks and designated shared tasks (Nivre et al., …

Content truncated to 500 words