MarSan at SemEval-2022 Task 11: Multilingual complex named entity recognition using T5 and transformer encoder

Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 1639 - 1647 July 14-15, 2022 ©2022 Association for Computational Linguistics MarSan at SemEval-2022 Task 11: Multilingual complex named entity recognition using T5 and transformer encoder Ehsan Tavan1,∗, Maryam Najafi1,∗ 1 NLP Department, Part AI Research Center, Tehran, Iran {ehsan.tavan, maryam.najafi}@partdp.ai Abstract The multilingual complex named entity recog- nition task of SemEval2020 required partici- pants to detect semantically ambiguous and complex entities in 11 languages. In order to participate in this competition, a deep learning model is being used with the T5 text-to-text lan- guage model and its multilingual version, MT5, along with the transformer’s encoder module. The subtoken check has also been introduced, resulting in a 4% increase in the model F1- score in English. We also examined the use of the BPEmb model for converting input tokens to representation vectors in this research. A performance evaluation of the proposed entity detection model is presented at the end of this paper. Six different scenarios were defined, and the proposed model was evaluated in each sce- nario within the English development set. Our model is also evaluated in other languages. 1 Introduction Named Entity Recognition (NER) is a key com- ponent of Natural Language Processing (NLP) as- signed to identify regions of text that contain refer- ences to entities. It is the process of identifying the informative part of data or applicable labels from unstructured data. In NER, data is gathered from unstructured data such as emails, blogs, newspa- pers, tweets, etc., to extract meaningful informa- tion. To put it another way, the term NER refers to identifying token spans of entities mentioned in the text and classifying them into a set of predeter- mined categories. The system finds entities from unstructured data and organizes them into multiple categories. As an extension of NLP, the field of NER can be considered as Information Extraction (IE). For NLP, IEs are among the trending fields and play an essential role in the following tasks: ∗Equal contribution. Listing order is random. Find and understand limited relevant parts of texts, Gather information from many pieces of text, and Produce the unified representation of all the rel- evant information. The NER problem falls into a general class of NLP problems called sequence tagging. Part of Speech (POS) tagging and chunk- ing are sequence tagging NLP tasks in addition to NER. It is possible to detect NER in three ways: flat NER, nested NER, and discontinuous NER. Nested NER has overlapping in the span of text Yan et al. (2021). Most approaches only target flat entities, ignoring nested structures common in many scenarios. It is challenging to identify spans as well as types of named entities in text Lample et al. (2016). So to find the best architecture, we implement a transformer-based language model in this research. NLP has significantly benefited from transfer learning in recent years. Transfer learning gains power and effectiveness from pre-training on large, unlabeled text datasets. The model can then be …

Content truncated to 500 words