Jack-Ryder at SemEval-2023 Task 5: Zero-Shot Clickbait Spoiling by Rephrasing Titles as Questions

Proceedings of the The 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 1090–1095 July 13-14, 2023 ©2023 Association for Computational Linguistics Jack-Ryder at SemEval-2023 Task 5: Zero-Shot Clickbait Spoiling by Rephrasing Titles as Questions Dirk Wangsadirdja Jan Pfister Konstantin Kobs Julius-Maximilians-Universität Würzburg (JMU)Andreas Hotho {surname}@informatik.uni-wuerzburg.de Abstract In this paper, we describe our approach to the clickbait spoiling task of SemEval 2023. The core idea behind our system is to leverage pre- trained models capable of Question Answering (QA) to extract the spoiler from article texts based on the clickbait title without any task- specific training. Since oftentimes, these titles are not phrased as questions, we automatically rephrase the clickbait titles as questions in or- der to better suit the pretraining task of the QA- capable models. Also, to fit as much relevant context into the model’s limited input size as possible, we propose to reorder the sentences by their relevance using a semantic similarity model. Finally, we evaluate QA as well as text generation models (via prompting) to ex- tract the spoiler from the text. Based on the validation data, our final model selects each of these components depending on the spoiler type and achieves satisfactory zero-shot results. The ideas described in this paper can easily be applied in fine-tuning settings. 1 Introduction The objective of the Clickbait Spoiling task (Fröbe et al., 2023a) is to find/extract the spoiler from a linked article that satisfies the reader’s curiosity that was induced by a clickbait title (e.g. “You won’t be- lieve how well this SemEval clickbait spoiling sys- tem performs!”). The challenge differentiates be- tween three types of spoilers: phrase (the spoiler is a short phrase), passage (a sentence or multiple sen- tences that span from a beginning position to an end position without any gap), and multi (the spoiler consists of multiple disjoint phrases/passages). In this paper, we describe our system for this challenge that is based only on pre-trained models and no fine-tuning. We model spoiler extraction as a Question Answering (QA) task and explore four different pre-trained architectures: Two extractive QA using BERT models and two prompt-based text generation models (with two prompts each). In order to make the clickbait title more similar to the questions the QA models have been trained on, we propose to automatically rephrase them into a question form. To ensure that the model receives the most relevant article text passages in its limited context, we propose to filter and sort the article text using a pre-trained semantic similarity model. We evaluate our pipeline on the validation dataset and combine the best components for each spoiler type as our final model. We find that rephrasing the clickbait title usually improves performance, while relevance-based context ordering does not. Our final model makes use of clickbait title rephrasing for phrase and multi-spoiler types, a DeBERTa QA model for phrase and passage-type spoilers, and a Flan-T5 model for multi-type spoilers. The results on the test set show satisfactory results, considering that our system is not fine-tuned in …

Content truncated to 500 words