SiNER: A Large Dataset for Sindhi Named Entity Recognition

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 2953–2961 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 2953 SiNER: A Large Dataset for Sindhi Named Entity Recognition Wazir Ali, Junyu Lu, Zenglin Xu School of Computer Science and Engineering University of Electronic Science and Technology of China, 611731 aliwazirjam, cs.junyu, zenglin.xu@gmail.com Abstract We introduce the SiNER: a named entity recognition (NER) dataset for low-resourced Sindhi language with quality baselines. It contains 1,338 news articles and more than 1.35 million tokens collected from Kawish and Awami Awaz Sindhi newspapers using the begin- inside-outside (BIO) tagging scheme. The proposed dataset is likely to be a significant resource for statistical Sindhi language processing. The ultimate goal of developing SiNER is to present a gold-standard dataset for Sindhi NER along with quality baselines. We implement several baseline approaches of conditional random field (CRF) and recent popular state-of-the-art bi-directional long-short term memory (Bi-LSTM) models. The promising F1-score of 89.16% outputted by the Bi-LSTM-CRF model with character-level representations demonstrates the quality of our proposed SiNER dataset. Keywords: Language Resources, SiNER, Sindhi Language, Named Entity Recognition 1. Introduction Named entity recognition is an essential lower-level task (Ma and Hovy, 2016) in natural language processing (NLP), used to extract and categorize naming entities into a predefined set of classes such as person, location, organization (Sang and De Meulder, 2003), numeral and temporal entities (dos Santos et al., 2015). It is essential to have a high-quality NER system for downstream NLP tasks such as information extraction (Grishman and Sundheim, 1996; Neudecker, 2016), question answering (Moldovan, 2002) and machine translation (Babych and Hartley, 2003). The NER task traditionally requires a large amount of knowledge in the form of lexicons and feature engineering to achieve high performance (Chiu and Nichols, 2016). The remarkable development has been made in the NER task since the message understanding conference (Grishman and Sundheim, 1996). Later, (Sang and Erik, 2002; Sang and De Meulder, 2003) introduced quality datasets for European languages along with exact match evaluation matrices. Numerous methods have been employed for the NER task, which can be broadly categorized into the rule-based, and language-independent statistical approaches. Recently, state- of-the-art language-independent deep learning models proposed by (Chiu and Nichols, 2016; Ma and Hovy, 2016; Lample et al., 2016; Tran et al., 2017; Kuru et al., 2016) have been successfully and extensively opted to address NER related problems with unsupervised word embeddings. But these language-independent neural models can be exploited on the substantial amount of training and evaluation datasets. The language resources (LRs) play an essential role in the digital survival of natural languages because of the ever- increasing usage of web-based technologies in daily life. Most of the European and East Asian languages are rich in such LRs, but most of the South and Southeast Asian Languages (SSALs) including Sindhi are still under- resourced (Ekbal et al., 2008; Singh, 2008). Sindhi has the status of an official language in the Sindh province of Pakistan and …

Content truncated to 500 words