CLaC at SemEval-2023 Task 2: Comparing Span-Prediction and Sequence-Labeling Approaches for NER

Proceedings of the The 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 1558–1561 July 13-14, 2023 ©2023 Association for Computational Linguistics CLaC at SemEval-2023 Task 2: Comparing Span-Prediction and Sequence-Labeling approaches for NER Harsh Verma, Sabine Bergler CLaC Labs, Concordia University {h_ver, bergler} @cse.concordia.ca Abstract This paper summarizes the CLaC submission for the MultiCoNER 2 task which concerns the recognition of complex, fine-grained named en- tities. We compare two popular approaches for NER, namely Sequence Labeling and Span Prediction. We find that our best Span Predic- tion system performs slightly better than our best Sequence Labeling system on test data. Moreover, we find that using the larger version of XLM RoBERTa significantly improves per- formance. Post-competition experiments show that Span Prediction and Sequence Labeling approaches improve when they use special in- put tokens (<s> and </s>) of XLM-RoBERTa. The code for training all models, preprocessing, and post-processing is available at this Github repo. 1 Introduction MultiCoNER 2 (Fetahu et al., 2023b,a) is the sec- ond version of the MultiCoNER (Malmasi et al., 2022) shared task. It focuses on identifying com- plex named entities in short sentences that lack context and addresses the challenge of identifying and distinguishing 30 fine-grained entity types and of handling simulated errors, such as typos. Competition data is tokenzied and tagged with the BIO scheme, i.e. the label B for begin, I for inside, and O for outside target entities. For this challenge, the 30 entity types were distinguished using suffixes to the B and I labels, e.g. B-Corp and I-Corp, as illustrated in Example 1. NER is frequently formulated as a sequence la- beling problem (Chiu and Nichols, 2016; Ma and Hovy, 2016; Wang et al., 2022), in which a model learns to label each token individually using the BIO labeling scheme. In contrast, span prediction approaches (Jiang et al., 2020; Li et al., 2020; Fu et al., 2021; Zhang et al., 2023) label entire text spans with entity types. Recently, (Fu et al., 2021) compared the two approaches and found that se- quence labeling performed better for long entities with low label consistency but that span prediction performed better for out of vocabulary words and entities of short to medium length. Because Multi- CoNER 2 tests both, the ability to identify out of vocabulary entities and entities of varying length1, it is interesting to compare the two approaches on this task. Our submission to the English Track does not use any external knowledge and holds rank 21 of 41 on Codalab2 with a macro-F1 of 59. 2 Submitted Systems 2.1 Tokenization The competition input consisted of tokenized data. The gold labels are given in form of BIO tags, where each token is labeled with a corresponding BIO tag as shown in Example 1. Sample input is given to XLM RoBERTa (Conneau et al., 2020) as the raw input string, without RoBERTa input tags such as <s> and </s> in the submission systems. (1) Token BIO-Tag Opharma Ocompany eli B-Corp lilly I-Corp and I-Corp company …

Content truncated to 500 words