Towards a Unified Multi-Domain Multilingual Named Entity Recognition Model

Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 2210–2219 May 2-6, 2023 ©2023 Association for Computational Linguistics Towards a Unified Multi-Domain Multilingual Named Entity Recognition Model Mayank Kulkarni∗ # 2, Daniel Preo¸tiuc-Pietro1, Karthik Radhakrishnan1, Genta Indra Winata1, Shijie Wu1, Lingjue Xie1, Shaohua Yang∗ 1Bloomberg 2Amazon Alexa AI {dpreotiucpie, gwinata}@bloomberg.net Abstract Named Entity Recognition is a key Natural Language Processing task whose performance is sensitive to choice of genre and language. A unified NER model across multiple gen- res and languages is more practical and effi- cient through leveraging commonalities across genres or languages. In this paper, we pro- pose a novel setup for NER which includes multi-domain and multilingual training and evaluation across 13 domains and 4 languages. We explore a range of approaches to building a unified model using domain and language adaptation techniques. Our experiments high- light multiple nuances to consider while build- ing a unified model, including that naive data pooling fails to obtain good performance, that domain-specific adaptations are more impor- tant than language-specific ones and that in- cluding domain-specific adaptations in a uni- fied model can reach performance close to train- ing multiple dedicated monolingual models at a fraction of their parameter count. 1 Introduction Identifying named entities, such as organization and people in text is a key NLP task situated up- stream of other NLP tasks such as co-reference resolution (Ratinov and Roth, 2012; Dutta and Weikum, 2015; Miwa and Bansal, 2016; Luo and Glass, 2018) or relation extraction (Nguyen and Gr- ishman, 2015; Zhong and Chen, 2021) and can en- hance applications including information retrieval (Carpineto and Romano, 2012; Berger and Laf- ferty, 2017) and summarization (Cheng and Lap- ata, 2016; Liu and Lapata, 2019; Maddela et al., 2022; Hofmann-Coyle et al., 2022). However, it has been established that NER is very sensitive to genre differences (Augenstein et al., 2017; Agar- wal et al., 2021),1 with models trained on one genre ∗Work done while at Bloomberg. #The authors are listed in alphabetical order. 1Throughout this paper, by genre we refer to a collection of documents with variations in style or structure that might Figure 1: A graphic comparison of performance of lan- guage, genre and joint adaptation by demo parameter count. performing poorly on a different one. As a result, multiple data sets were created to allow domain- specific models to be trained. Yet, domain adapta- tion especially across multiple genres has shown the promise that a single model could improve per- formance on multiple domains (Wang et al., 2020; Liu et al., 2021). Further, the advent of pretrained multilingual models (Wu and Dredze, 2019; Con- neau et al., 2020) enables transfer across languages in an straightforward way by simply feeding het- erogeneous language data in fine-tuning, making cross-lingual training feasible and a new dimen- sion of adaptation available for exploration. Thus, performance on a specific genre and language pair could be improved by leveraging commonalities in training data across both genre and language dimen- …

Content truncated to 500 words