Don’t Throw Those Morphological Analyzers Away Just Yet: Neural Morphological Disambiguation for Arabic

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 704–713 Copenhagen, Denmark, September 7–11, 2017. c⃝2017 Association for Computational Linguistics Don’t Throw Those Morphological Analyzers Away Just Yet: Neural Morphological Disambiguation for Arabic Nasser Zalmout Nizar Habash Computational Approaches to Modeling Language Lab New York University Abu Dhabi United Arab Emirates {nasser.zalmout,nizar.habash}@nyu.edu Abstract This paper presents a model for Arabic morphological disambiguation based on Recurrent Neural Networks (RNN). We train Long Short-Term Memory (LSTM) cells in several configurations and embed- ding levels to model the various morpho- logical features. Our experiments show that these models outperform state-of-the- art systems without explicit use of feature engineering. However, adding learning features from a morphological analyzer to model the space of possible analyses pro- vides additional improvement. We make use of the resulting morphological mod- els for scoring and ranking the analyses of the morphological analyzer for morpho- logical disambiguation. The results show significant gains in accuracy across sev- eral evaluation metrics. Our system re- sults in 4.4% absolute increase over the state-of-the-art in full morphological anal- ysis accuracy (30.6% relative error reduc- tion), and 10.6% (31.5% relative error re- duction) for out-of-vocabulary words. 1 Introduction Recurrent Neural Networks (RNN) in general, and Long Short-Term Memory (LSTM) cells in par- ticular, have been proven very successful for vari- ous Natural Language Processing (NLP) tasks, es- pecially those involving sequential data tagging. RNN models can produce near or above state- of-the-art performance with minimal language- specific feature engineering. These models have the capacity of capturing syntactic and seman- tic features through the lexical word-level embed- dings, and subword features through character- level embeddings. Morphologically rich languages pose many challenges to NLP through their high degree of ambiguity and sparsity. These challenges are ex- acerbated for languages with limited resources. Morphological analyzers help reduce sparsity by providing several out-of-context morpheme-based analyses for words, but they usually introduce am- biguity by returning multiple analyses for the same surface form. Therefore, the model would require a further step of morphological disambiguation to choose the correct analysis in context. Morphological modeling involves heavy use of sequential tagging, so using an LSTM-based model would be highly advantageous. LSTM models are also optimal for long-sequence tagging in particular, so such systems should be able to outperform other deep learning models with fixed window-based modeling. Morphological disam- biguation is a well studied problem in the litera- ture, but LSTM-based contributions are still rela- tively scarce. In this paper we use Bidirectional- LSTM (Bi-LSTM) models for morphological tag- ging and language modeling, and use the results of these models in ranking the analyses of the mor- phological analyzer. We incorporate various sub- word and morphological features at different lin- guistic depths in the tagger, along with both word- based and character-based embeddings. Our results show significant accuracy gains for all the morphological features we study, and across several evaluation metrics. We compare our system against a strong baseline and a state- of-the-art-system. We achieve 4.4% absolute over the …

Content truncated to 500 words