Headline Token-based Discriminative Learning for Subheading Generation in News Article

Findings of the Association for Computational Linguistics: EACL 2023, pages 2128–2135 May 2-6, 2023 ©2023 Association for Computational Linguistics Headline Token-based Discriminative Learning for Subheading Generation in News Article Joonwon Jang Sejong University / Republic of Korea joonwon.lainshower@gmail.com Misuk Kim∗ Sejong University / Republic of Korea mskim.sju@gmail.com Abstract The news subheading summarizes an article’s contents in several sentences to support the headline limited to solely conveying the main contents. So, it is necessary to generate com- pelling news subheadings in consideration of the structural characteristics of the news. In this paper, we propose a subheading generation model using topical headline information. We introduce a discriminative learning method that utilizes the prediction result of masked headline tokens. Experiments show that the proposed model is effective and outperforms the compar- ative models on three news datasets written in two languages. We also show that our model performs robustly on a small dataset and vari- ous masking ratios. Qualitative analysis and hu- man evaluations also show that the overall qual- ity of generated subheadings improved over the comparative models. 1 Introduction The news headline summarizes the article to grab the attention and interest of the readers (Dor, 2003; Ifantidou, 2009; Ecker et al., 2014). However, the headline is written in a brief form of short sen- tences with topic-related phrases (Yamada et al., 2021), making it hard for users to grasp the entire content of the news article from the headline alone. To tackle this problem, some news vendors pro- vide a subheading, usually located right below the headline, to convey its main content within several sentences. This component can provide a core and informative summary of a news article that cannot be conveyed by the headline alone. Mainly, sub- headings are written by professional news writers with concise content that corresponds to the main body of the news. Recently, Hasan et al. (2021) released XLSum, a multilingual news summary dataset, referring to subheading as a summary of the article. Therefore, generating subheadings can be considered an ab- stractive summarization problem that needs to cap- ture the topical knowledge from the body of the article. The main approach is to add auxiliary sig- nals to make the model aware of topical knowledge. Dou et al. (2021) and Aralikatte et al. (2021) add an external guidance signal by lexical similarity be- tween input text and summary. Yamada et al. (2021) extracts the context word sequences from the refer- ence to reflect some important phrases from the ar- ticle. Although these external auxiliary sources pro- vide diverse topical signals, they are cost-intensive to heuristically manipulate and have limitations in guiding the overall topical information of the arti- cle. Other approaches incorporate contrastive learn- ing into sequence-to-sequence (seq2seq) model, allowing the model to learn topical representation of the input text (Lee et al., 2021; Liu et al., 2021; Wu et al., 2020). They explicitly constructs positive or negative inputs to introduce contrastive loss as an augmentation of MLE training. In this work, we propose a novel …

Content truncated to 500 words