Billie-Newman at SemEval-2023 Task 5: Clickbait Classification and Question Answering with Pre-Trained Language Models, Named Entity Recognition and Rule-Based Approaches

Proceedings of the The 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 1542–1550 July 13-14, 2023 ©2023 Association for Computational Linguistics Billie-Newman at SemEval-2023 Task 5: Clickbait Classification and Question Answering with Pre-Trained Language Models, Named Entity Recognition and Rule-Based Approaches Andreas Kruff and Anh Huy Matthias Tran Claudiusstraße 1, 50678 Köln andreas.kruff@smail.th-koeln.de anh_huy_matthias.tran@smail.th-koeln.de Abstract In this paper, we describe the implementations of our systems for the SemEval-2023 Task 5 ‘Clickbait Spoiling‘, which involves the clas- sification of clickbait posts in sub-task 1 and the spoiler generation and question answer- ing of clickbait posts in sub-task 2, ultimately achieving a balanced accuracy of 0.593 and a BLEU score of 0.322 on the test datasets in sub-task 1 and sub-task 2 respectively. For this, we propose the usage of RoBERTa trans- former models and modify them for each spe- cific downstream task. In sub-task 1, we use the pre-trained RoBERTa model and use it in conjunction with NER, a spoiler-title ratio, a regex check for enumerations and lists and in- put reformulation. In sub-task 2, we propose the usage of the RoBERTa-SQuAD2.0 model for extractive question answering in combina- tion with a contextual rule-based approach for multi-type spoilers in order to generate spoiler answers. 1 Introduction Clickbait involves a text, usually accompanied by a thumbnail or a link, specifically designed to invoke the interest or curiosity of a user. This is done with the intent of enticing the user into clicking the text in order to satisfy their created curiosity. In Task 5 from SemEval-2023, the Clickbait Challenge engages with the study of multi-class classification and extractive question answering in respect to these aforementioned clickbait posts (Hagen et al., 2022). For this purpose, the task suggests the act of clickbait spoiling, which involves the generation of a short text that answers this supposed curiosity gap (Fröbe et al., 2023a). While models engaging in clickbait spoiling can be used for the purpose of saving a user a click on clickbait posts, the application of models able to question answer a set of questions reliably extends to many more use cases such as answering search requests formed as questions in classic information retrieval, making insights gained from question answering models valuable. In order to facilitate the study, task 5 is di- vided into two subtasks: Sub-task 1: Spoiler Type Classification This sub-task involves the automated classification of clickbait posts into the category of • phrase: a post that can be answered with a single phrase such as persons, dates or organi- sations • passage: a post that requires an entire passage to be answered • multi: a post that can be answered with a list or an enumeration of items Therefore, sub-task 1 can be identified as a multi-class classification problem. Sub-task 2: Spoiler Generation This sub-task involves satisfying the curiosity of a proposed user via question answering. This is done by inspecting the clickbait post and the linked content for relevant passages and using those passages to generate the spoiler answer, making …

Content truncated to 500 words