BERT-PersNER: A New Model for Persian Named Entity Recognition

Proceedings of Recent Advances in Natural Language Processing, pages 647–654 Sep 1–3, 2021. https://doi.org/10.26615/978-954-452-072-4_073 647 BERT-PersNER: a New Model for Persian Named Entity Recognition Farane Jalali Farahani Department of Computer Engineering Sharif University of Technology Tehran, Iran fjalali@ce.sharif.edu Gholamreza Ghassem-Sani Department of Computer Engineering Sharif University of Technology Tehran, Iran sani@sharif.edu Abstract Named entity recognition (NER) is one of the major tasks in natural language pro- cessing. A named entity is often a word or expression that bears a valuable piece of information, which can be effectively em- ployed by some major NLP tasks such as machine translation, question answering, and text summarization. In this paper, we introduce a new model called BERT- PersNER (BERT based Persian Named En- tity Recognizer), in which we have applied transfer learning and active learning ap- proaches to NER in Persian, which is re- garded as a low-resource language. Like many others, we have used Conditional Random Field for tag decoding in our pro- posed architecture. BERT-PersNER has outperformed two available studies in Per- sian NER, in most cases of our experi- ments using the supervised learning ap- proach on two Persian datasets called Ar- man and Peyma. Besides, as the very first effort to try active learning in the Persian NER, using only 30% of Arman and 20% of Peyma, we respectively achieved 92.15%, and 92.41% performance of the mentioned supervised learning experiments. 1 Introduction Named Entity Recognition (NER) is a funda- mental Natural Language Processing (NLP) task, in which we try to identify and classify certain entities such as organizations, persons, locations, etc. in a given text. A named entity is often a word or expression in the text that bears a valuable piece of information, which can be effectively used in other high-level NLP tasks such as machine translation, question an- swering, and text summarization. Unlike En- glish, which is rich in digital resources, there are some natural languages such as Persian, which is regarded to be a low-resource lan- guage. However, Persian is an important lan- guage because it is spoken by more than 110 million people worldwide. In this article, we introduce an efficient model for Persian NER. To tackle problems such as the lack of labeled data in Persian, we have used two different learning methods, i.e., transfer learning (Pan and Yang, 2010) and active learning (Settles, 2010). To trans- fer knowledge in transfer learning, the model- based (parameter) approach (Settles, 2010) has been used to fine-tune BERT (Devlin et al., 2019). Through transfer learning, we can pro- duce a powerful model using limited data that takes much less training time. We also em- ployed active learning in order to perform the fine-tuning task by using only a few infor- mative samples instead of the whole dataset. Through active learning, it is possible to de- liver a performance that is very close to that of a supervised learning method. The structure of this paper is as follows. We review the literature in Sec. 2; BERT is dis- cussed …

Content truncated to 500 words