silpa_nlp at SemEval-2022 Tasks 11: Transformer based NER models for Hindi and Bangla languages

Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 1536 - 1542 July 14-15, 2022 ©2022 Association for Computational Linguistics silpa_nlp at SemEval-2022 Tasks 11: Transformer based NER models for Hindi and Bangla languages Pawankumar Jawale∗ Sumit Singh ∗ Uma Shanker Tiwary Indian Institute of Information Technology, Allahabad, UP, India {pawankumar.jawale, sumitrsch}@gmail.com ust@iiita.ac.in Abstract We present Transformer based pretrained mod- els, which are fine-tuned for Named Entity Recognition (NER) task. Our team par- ticipated in SemEval-2022 Task 11 Multi- CoNER: Multilingual Complex Named En- tity Recognition task for Hindi and Bangla. Result comparison of six models (mBERT, IndicBERT, MuRIL (Base), MuRIL (Large), XLM-RoBERTa (Base) and XLM-RoBERTa (Large) ) has been performed. It is found that among these models MuRIL (Large) model performs better for both the Hindi and Bangla languages. Its F1-Scores for Hindi and Bangla are 0.69 and 0.59 respectively. 1 Introduction Named Entity Recognition (NER) is one of the hot topic in natural language processing (NLP). NER is a task of identification of named entities from given sentence and their classification into predefined classes like Person, Location, Organi- sation, Corporation etc. For below sentence: राम �दल्ली में गूगल में काम करता है।, राम is Person, �दल्ली is Location and गूगल is Corporation. The application of NER can be found in other NLP tasks such as text summarization (Toda and Kataoka, 2005), information retrieval, machine translation (Babych and Hartley, 2003), question-answering (Molla Aliod et al., 2009). The researchers have come up with many ap- proaches for NER task such as Rule-based (Krupka and IsoQuest, 2005), feature-based Supervised approach (Liao and Veeramachaneni, 2009), Unsupervised approach and Deep learning based approach (Li et al., 2020) and Transformer based approach (Vaswani et al., 2017). ∗ Authors equally contributed to this work. The Transformer models are good at capturing features from lengthy sentences compared to re- current neural networks (Vaswani et al., 2017). RoBERTa model (Liu et al., 2019) performed good for NER task for rich resource languages like English. For Hindi and Bangla languages, we applied XLM-RoBERTa, which is a multilingual version of RoBERTa pre-trained in 100 languages (in- cluding Hindi and Bangla). We also applied In- dicBERT (Kakwani et al., 2020) and mBERT (De- vlin et al., 2018). At last, we applied MuRIL (Khanuja et al., 2021; Sharma et al., 2022) which is specifically pre-trained in the text of 17 Indic languages and it gave better result than above mod- els for the NER task. This paper consists of a total of six sections apart from the introduction. Section 2 briefly de- fines the problem definition and task provided by organizers. Section 3 discusses the work done till now on the NER task. Section 4 mentions the dataset being used in this paper. Section 5 de- scribes the general Transformer architecture for the NER task, along with prepossessing and post- processing. Section 6 discusses the results ob- tained by used models on both Hindi and Bangla languages and the error analysis. Finally, Section 7 concludes this paperwork. 2 Problem …

Content truncated to 500 words