Jack-flood at SemEval-2023 Task 5:Hierarchical Encoding and Reciprocal Rank Fusion-Based System for Spoiler Classification and Generation

Proceedings of the The 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 1906–1915 July 13-14, 2023 ©2023 Association for Computational Linguistics jack-flood at SemEval-2023 Task 5: Hierarchical Encoding and Reciprocal Rank Fusion-Based System for Spoiler Classification and Generation Sujit Kumar, Aditya Sinha∗, Soumyadeep Jana∗, Rahul Mishra and Sanasam Ranbir Singh Indian Institute of Technology Guwahati University of Geneva {sujitkumar, aditya2001, sjana, ranbir}@iitg.ac.in, rahul.mishra@unige.ch Abstract The rise of social media has exponentially witnessed the use of clickbait posts that grab users’ attention. This paper presents our ap- proach, which use different encoding tech- niques. We propose hierarchical encoding with count and document length feature-based model for spoiler type classification, which uses Recurrence over Pretrained Encoding. We also propose combining multiple ranking with reciprocal rank fusion for passage spoiler re- trieval and question-answering approach for phrase spoiler retrieval. For multipart spoiler retrieval, we combine the above two spoiler re- trieval methods. Based on the benchmark, the experimental results indicate that our proposed spoiler retrieval methods are highly effective in retrieving spoilers that are semantically very close to the ground truth spoilers. 1 Introduction The rapid adoption of social media among the masses has pushed online publishers to stay relevant by making their content prominent on these platforms (Teixeira, 2014). This is mostly achieved through clickbait. This paper presents our proposed approach and detailed analysis for two subtasks of the clickbait spoiling challenge (Task 5) (Fröbe et al., 2023) Subtask 1: clickbait spoiler type classification, b) Subtask 2: clickbait spoiler generation. This contest is organized in the 17th International Workshop on Semantic Evaluation (SemEval 2023). This challenge addresses the issue of generating spoilers for clickbait. The importance of this task can be gauged from the fact that spoilers would preemptively warn the readers about the content of the clickbait posts, enabling the users to make an informed decision. This would also mitigate the spread of misinformation (Silverman, 2015) and the over-dependence of con- tent publishers on sensational posts for prominence. ∗* Equal contributions Our proposed methods for spoiler-type classifi- cation and spoiler generation rely on strength of pretrained language models such as BERT (Devlin et al., 2018), and its different variants like DeBERTa (He et al., 2021). We propose a hierarchical encoding-based model for spoiler-type classifications (subtask-1), which encodes different aspects of hierarchical information presented in the target paragraph along with document length and count overlap-based features. Considering the unique challenges and characteristics posed by passage, phrase and multipart spoiler generation, we adopt three different approaches for spoiler generation. We combine several ranking models using reciprocal rank fusion (RRF) for passage retrieval, a pretrained RoBERTa model over question answering task for phrase retrieval and combine methods of passage retrieval and phrase retrieval for multipart spoiler retrieval. Our code repository is available at1https://github.com/ thesujitkumar/jack-flood-at-SemEval-2023-Task-5_ Spoiler-Classification-and-Generation to recreate and reproduce the results presented in this paper. 2 Background Initial studies (Blom and Hansen, 2015; Schaffer, 1995; Rony et al., 2017; Anand et al., 2017; Biyani et al., 2016; Agrawal, 2016; Chakraborty et al., 2016) in …

Content truncated to 500 words