PA Ph&Tech at SemEval-2022 Task 11: NER Task with Ensemble Embedding from Reinforcement Learning

Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 1444 - 1447 July 14-15, 2022 ©2022 Association for Computational Linguistics PA Ph&Tech at SemEval-2022 Task 11: NER Task with Ensemble Embedding from Reinforcement Learning Qizhi LIN, Xiaopeng WANG, Xiandi JIANG, Benqi WANG, Qifeng XIAO Ping An Puhui Enterprise Management Co. Ltd, Shanghai, China linqizhi090@lu.com, wangxiaopeng069@lu.com Changyong HOU, Yixuan QIAO, Jun WANG, Peng JIANG Ping An Healthcare Technology, Beijing, China wangjun916@pingan.com.cn Abstract From pretrained contextual embedding to document-level embedding, the selection and construction of embedding have drawn more and more attention in the NER domain in recent research. This paper aims to discuss the per- formance of ensemble embeddings on complex NER tasks. Enlightened by Wang’s methodol- ogy, we try to replicate the dominating power of ensemble models with reinforcement learning optimizor on plain NER tasks to complex ones. Based on the composition of semeval dataset, the performance of the applied model is tested on lower-context, QA, and search query scenar- ios together with its zero-shot learning ability. Results show that with abundant training data, the model can achieve similar performance on lower-context cases compared to plain NER cases, but can barely transfer the performance to other scenarios in the test phase. 1 Introduction Named Entity Recognition (NER) as a typical topic in the NLP field, has displayed numerous outstand- ing outcomes in recent years, especially benefiting from development of pretrained models. However, there are still challenging aspects that remain to be tackled, such as short texts (low-context), emerging entities, and complex entities. In this task, corpus from low-context, QA and search query scenarios are collected to represent different complex NER tasks (Meng et al., 2021). Wang et al. (2021) pro- posed an automated concatenation model on plain NER tasks with benchmark dataset like CONLL03, which automatically generates optimized concate- nation of stack embeddings with reinforcement learning strategies for structure prediction tasks like NER. A similar methodology is applied on the given dataset to test if the challenges mentioned above can be properly addressed. 2 Related Work The development of sequence tagging models can be concluded mainly into two separate parts: en- coder and decoder. Ever since the BiLSTM-CRF model (Ma and Hovy, 2016) was brought up, it has been widely used as the decoder end of NER models, while researchers shifted increasing atten- tion to the structure of encoder. Various categories of embeddings have been raised, including char- acter embeddings, non-contextual embedding like word2vec (Mikolov et al., 2013) and pretrained con- textualized embeddings like ELMo (Peters et al., 2018) and Flair (Akbik et al., 2018). With the emer- gence of Transformers, the performance of large pretrained contextualized models (Devlin et al., 2019)have sweeped the leaderboard in lots of NLP tasks. In terms of language, Multilingual BERT(M- BERT) (Pires et al., 2019) demonstrates excellent representation of multilingual embeddings, which was later surpassed by XLM-R (Conneau et al., 2020), a more powerful and comprehensive mul- tilingual model. To better allocate these embed- ding methods, many recent researchers have tried different …

Content truncated to 500 words