DAMO-NLP at SemEval-2023 Task 2: A Unified Retrieval-augmented System for Multilingual Named Entity Recognition

Proceedings of the The 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 2014–2028 July 13-14, 2023 ©2023 Association for Computational Linguistics DAMO-NLP at SemEval-2023 Task 2: A Unified Retrieval-augmented System for Multilingual Named Entity Recognition Zeqi Tan�†, Shen Huang⋆†, Zixia Jia☼†, Jiong Cai☼†, Yinghui Li♥†, Weiming Lu� Yueting Zhuang�, Kewei Tu☼, Pengjun Xie⋆, Fei Huang⋆, Yong Jiang⋆∗ ⋆DAMO Academy, Alibaba Group �College of Computer Science and Technology, Zhejiang University ☼School of Information Science and Technology, ShanghaiTech University ♥Tsinghua Shenzhen International Graduate School, Tsinghua University {zqtan,yzhuang,luwm}@zju.edu.cn liyinghu20@mails.tsinghua.edu.cn {jiazx,caijiong,tukw}@shanghaitech.edu.cn {pangda,chengchen.xpj,f.huang,yongjiang.jy}@alibaba-inc.com Abstract The MultiCoNER II shared task aims to tackle multilingual named entity recognition (NER) in fine-grained and noisy scenarios, and it in- herits the semantic ambiguity and low-context setting of the MultiCoNER I task. To cope with these problems, the previous top systems in the MultiCoNER I either incorporate the knowl- edge bases or gazetteers. However, they still suffer from insufficient knowledge, limited con- text length, single retrieval strategy. In this paper, our team DAMO-NLP proposes a uni- fied retrieval-augmented system (U-RaNER) for fine-grained multilingual NER. We perform error analysis on the previous top systems and reveal that their performance bottleneck lies in insufficient knowledge. Also, we discover that the limited context length causes the retrieval knowledge to be invisible to the model. To en- hance the retrieval context, we incorporate the entity-centric Wikidata knowledge base, while utilizing the infusion approach to broaden the contextual scope of the model. Also, we ex- plore various search strategies and refine the quality of retrieval knowledge. Our system1 wins 9 out of 13 tracks in the MultiCoNER II shared task. Additionally, we compared our system with ChatGPT, one of the large lan- guage models which have unlocked strong ca- pabilities on many tasks. The results show that there is still much room for improvement for ChatGPT on the extraction task. 1 Introduction The MultiCoNER series shared task (Malmasi et al., 2022b; Fetahu et al., 2023b) aims to iden- tify complex named entities (NE), such as titles ∗: project lead. †: equal contributions. This work was done during Zeqi Tan, Zixia Jia, Jiong Cai, and Yinghui Li’s internship at DAMO Academy, Alibaba Group. 1We will release the dataset, code, and scripts of our system at https://github.com/modelscope/ AdaSeq/tree/master/examples/U-RaNER. It is first conceptualized by Erving Goffman in 1959. Wrong prediction in RaNER: Input with gold annotations: It is first conceptualized by Erving Goffman in 1959. Knowledge used in RaNER: Goffman's The Presentation of Self in Everyday Life was published in 1956, with a revised edition in 1959. He had developed the book's core ideas from his doctoral dissertation. Artist OtherPER Figure 1: An example of wrong prediction in RaNER (Wang et al., 2022b) (one of the top systems in the MultiCoNER I task (Malmasi et al., 2022b)) . This case illustrates that the knowledge covered is not sufficient for fine-grained complex NER. of creative works, which do not possess the tra- ditional characteristics of named entities, such as persons, locations, etc. It is challenging to identify these ambiguous …

Content truncated to 500 words