Performance Analysis of Arabic Pre-trained Models on Named Entity Recognition Task

Proceedings of Recent Advances in Natural Language Processing, pages 458–467 Varna, Sep 4–6, 2023 https://doi.org/10.26615/978-954-452-092-2_051 458 Performance Analysis of Arabic Pre-Trained Models on Named Entity Recognition Task Abdelhalim Hafedh Dahou, Mohamed Amine Cheragui, Ahmed Abdelali GESIS – Leibniz-Institute for the Social Sciences, Cologne, Germany Mathematics and Computer Science Department, Ahmed Draia University, Adrar, Algeria Qatar Computing Research Institute, Hamad Bin Khalifa University, Qatar abdelhalim.dahou@gesis.org m cheragui@univ-adrar.edu.dz aabdelali@hbku.edu.qa Abstract Named Entity Recognition (NER) is a crucial task within natural language processing (NLP) that entails the identification and classification of entities such as person, organization and lo- cation. This study delves into NER specifi- cally in the Arabic language, focusing on the Algerian dialect. While previous research in NER has primarily concentrated on Modern Standard Arabic (MSA), the advent of social media has prompted a need to address the variations found in different Arabic dialects. Moreover, given the notable achievements of Large-scale pre-trained models (PTMs) based on the BERT architecture, this paper aims to evaluate Arabic pre-trained models using an Algerian dataset that covers different domains and writing styles. Additionally, an error anal- ysis is conducted to identify PTMs’ limita- tions, and an investigation is carried out to as- sess the performance of trained MSA models on the Algerian dialect. The experimental re- sults and subsequent analysis shed light on the complexities of NER in Arabic, offering valu- able insights for future research endeavors. 1 Introduction The expression named entities recognition (NER) has been used for the first time at the 6th edi- tion of the Message Understanding Conference (MUC) in November 1995 (Grishman and Sund- heim, 1996). The task of NER consisted in using SGML markers to identify entities in texts (names of persons, organizations, or places), temporal ex- pressions, and numerical expressions (”currency” or ”percentages”). Since then, NER has become a starting point and an important part of many appli- cations in natural language processing (Ali et al., 2020), such as: Information Extraction (IE) (Ku- mar and Starly, 2022), Information Retrieval (IR) (Guo et al., 2009), Semantic Annotation (SA) (Li et al., 2022), Machine Translation (MT) (Babych and Hartley, 2003), Question Answering (QA) sys- tems (Yadav and Bethard, 2018), Text Summariza- tion (Aone, 1999) and Text Clustering (Nagrale et al., 2019). The process of NER can be done according to three main approaches (Oudah and Shaalan, 2017) (Mansouri et al., 2008), (Gorinski et al., 2019): the symbolic or linguistic (rule-based) approach, where the main idea is to use linguistic knowledge (inter- nal or external clues), dictionaries and gazetteers of proper names to establish a list of knowledge rules (called regular expressions or finite state transduc- ers (Mesfar, 2007)). However, the principal incon- venience of this approach is that the rule-generation process is fastidious and time-consuming. The Ma- chine Learning (ML) approach, is mainly based on a previously annotated corpus. where the recog- nition problem is converted into a classification problem and employs various ML models to solve it. A hybrid approach which combines the two pre- vious approaches to boost …

Content truncated to 500 words