HiNER: A large Hindi Named Entity Recognition Dataset

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 4467–4476 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 4467 HiNER: A Large Hindi Named Entity Recognition Dataset Rudra Murthy2, Pallab Bhattacharjee1, Rahul Sharnagat3, Jyotsana Khatri1, Diptesh Kanojia4, Pushpak Bhattacharyya1 1CFILT Lab, IIT Bombay, India. 2IBM IRL, Bangalore, India. 3Walmart Labs, USA. 4Surrey Institute for People-centred AI, University of Surrey, United Kingdom. 1bhattacharjee.pallab9@gmail.com, {jyotsanak, pb}@iitb.ac.in 2rmurthyv@in.ibm.com, 3rdsharnagat@gmail.com, 4d.kanojia@surrey.ac.uk Abstract Named Entity Recognition (NER) is a foundational NLP task that aims to provide class labels like Person, Location, Organisation, Time, and Number to words in free text. Named Entities can also be multi-word expressions where the additional I-O-B annotation information helps label them during the NER annotation process. While English and European languages have considerable annotated data for the NER task, Indian languages lack on that front- both in terms of quantity and following annotation standards. This paper releases a significantly sized standard-abiding Hindi NER dataset containing 109,146 sentences and 2,220,856 tokens, annotated with 11 tags. We discuss the dataset statistics in all their essential detail and provide an in-depth analysis of the NER tag-set used with our data. The statistics of tag-set in our dataset show a healthy per-tag distribution, especially for prominent classes like Person, Location and Organisation. Since the proof of resource-effectiveness is in building models with the resource and testing the model on benchmark data and against the leader-board entries in shared tasks, we do the same with the aforesaid data. We use different language models to perform the sequence labelling task for NER and show the efficacy of our data by performing a comparative evaluation with models trained on another dataset available for the Hindi NER task. Our dataset helps achieve a weighted F1 score of 88.78 with all the tags and 92.22 when we collapse the tag-set, as discussed in the paper. To the best of our knowledge, no available dataset meets the standards of volume (amount) and variability (diversity), as far as Hindi NER is concerned. We fill this gap through this work, which we hope will significantly help NLP for Hindi. We release this dataset with our code and models for further research. Keywords: named entity recognition, dataset, Hindi, human-annotated, low-resource language 1. Introduction Named Entity Recognition (NER) is an essential lower- level task (Ma and Hovy, 2016) in Natural Language Processing (NLP), used to extract and categorize nam- ing entities into a predefined set of classes such as per- son, location, organization, numeral and temporal en- tities. A well-performing NER system can help the downstream tasks of Machine Translation (Babych and Hartley, 2003), Information Extraction (Neudecker, 2016), and Questions Answering (Moldovan and Sur- deanu, 2002). With the recent surge in the NER re- search (Sohrab and Miwa, 2018; Plank, 2019; Co- para et al., 2020; Grancharova and Dalianis, 2021), the NLP community has also created large annotated datasets for the NER task (Ali et al., 2020; Ding et al., 2021) including code-mixed datasets (Singh …

Content truncated to 500 words