Arabic Named Entity Recognition Using Variant Deep Neural Network Architectures and Combinatorial Feature Embedding Based on CNN, LSTM and BERT

Arabic Named Entity Recognition Using Variant Deep Neural Network Architectures and Combinatorial Feature Embedding Based on CNN, LSTM and BERT Manel Affi RIADI / ENSI University of Manouba / Tunis Tunisia manel.affi@ensi-uma.tn Chiraz Latiri LIPAH / FST University of Tunis El Manar/ Tunis Tunisia chiraz.latiri@gnet.tn Abstract One of the most important factors that sig- nificantly affects the quality of sequence la- beling models is the selection and encod- ing of input features. The complexity and morphological richness of Arabic language is the main reason why most existing Ara- bic Named Entity Recognition (NER) systems rely heavily on hand-crafted feature engineer- ing. To overcome such a limitation, we pro- pose a novel neural network architecture to tackle the Arabic NER task. The proposed ap- proach takes advantage of the recent success of deep neural networks in various Natural Language Processing (NLP) applications. We use a variant deep neural network architecture combined with a combinatorial feature em- bedding based on Convolutional Neural Net- work (CNN), Long Short-Term Memory net- works(LSTM) and BERT to generate rich se- mantic and syntactic word representation vec- tors. Without using any external knowledge or hand-crafted feature engineering, the pro- posed models outperform the state-of-the-art systems on the ANERCorp dataset by yield- ing an F1-score of 93.34% and 93.68 % using bidirectional LSTM-CRF (BLC) and bidirec- tional GRU-CRF (BGC) architectures, respec- tively. 1 Introduction Named Entity Recognition (NER) is defined as an information extraction task which aims to identify, extract and automatically categorize named entities into a set of predefined categories such as persons, organizations, locations, etc (Nadeau and Sekine, 2007). It is also considered as a mandatory pre- processing module in many wider NLP applications, such as information retrieval, speech recognition, syntactic parsing, machine translation, text classifi- cation, question answering, entity coreference reso- lution and entity linking and disambiguation. There is a fair amount of literature on NER research in English, Chinese and other widely spoken lan- guages. The massive growth of Arabic content on the web has led to increased demand for develop- ing accurate and robust Arabic NLP tools. In recent years, Arabic NER has become a challenging task and has received increasing attention from current researchers due to its characteristics and peculiari- ties (Farghaly and Shaalan, 2009), and given the lim- ited availability of annotated datasets (Zirikly and Diab, 2015). Researchers interested in Arabic NER have mainly pursued three approaches: rule-based (Mesfar, 2007; Zaghouani, 2012), machine learning-based (Benajiba and Rosso, 2008) and hybrid methods (Abdallah et al., 2012; Oudah and Shaalan, 2017). All these three methods suffer from the same prob- lem, as it requires a lot of language-specific knowl- edge and feature engineering to obtain useful results. This is even more highlighted by the deficiency of linguistic resources and the complex morphology and syntax of the language. Recently, the deep learning paradigm (Mishra and Gupta, 2017) has emerged and led to impressive ad- vances in fields such as speech processing (Oord et al., 2016) and image recognition (Hu et al., 2018). For NLP, …

Content truncated to 500 words