WR-One2Set: Towards Well-Calibrated Keyphrase Generation

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 7283 - 7293 December 7-11, 2022 ©2022 Association for Computational Linguistics WR-ONE2SET: Towards Well-Calibrated Keyphrase Generation Binbin Xie1,3, Xiangpeng Wei2, Baosong Yang2, Huan Lin2, Jun Xie2, Xiaoli Wang3, Min Zhang4 and Jinsong Su1,3∗ 1School of Informatics, Xiamen University, China 2Alibaba Group, China 3Key Laboratory of Digital Protection and Intelligent Processing of Intangible Cultural Heritage of Fujian and Taiwan, Ministry of Culture and Tourism, China 4Soochow University, China xdblb@stu.xmu.edu.cn pemywei@gmail.com yangbaosong.ybs@alibaba-inc.com sjs@xmu.edu.cn Abstract Keyphrase generation aims to automatically generate short phrases summarizing an input document. The recently emerged ONE2SET paradigm (Ye et al., 2021) generates keyphrases as a set and has achieved competitive perfor- mance. Nevertheless, we observe serious cal- ibration errors outputted by ONE2SET, espe- cially in the over-estimation of ∅ token (means “no corresponding keyphrase”). In this paper, we deeply analyze this limitation and identify two main reasons behind: 1) the parallel gener- ation has to introduce excessive ∅ as padding tokens into training instances; and 2) the train- ing mechanism assigning target to each slot is unstable and further aggravates the ∅ to- ken over-estimation. To make the model well- calibrated, we propose WR-ONE2SET which extends ONE2SET with an adaptive instance- level cost Weighting strategy and a target Re- assignment mechanism. The former dynam- ically penalizes the over-estimated slots for different instances thus smoothing the uneven training distribution. The latter refines the orig- inal inappropriate assignment and reduces the supervisory signals of over-estimated slots. Ex- perimental results on commonly-used datasets demonstrate the effectiveness and generality of our proposed paradigm. 1 Introduction Keyphrases are short phrases fully encoding the main information of a given document. They can not only facilitate readers to quickly understand the document, but also provide useful information to many downstream tasks, including document classification (Hulth and Megyesi, 2006), summa- rization (Wang and Cardie, 2013), etc. With the rapid development of deep learning, keyphrase generation (Meng et al., 2017) has at- tracted increasing attention due to its ability to pro- duce phrases that even do not match any contiguous ∗* Corresponding author. … Training Stage slot1 slot2 slot4 slot3 slotN Predictions: Assigned Targets: Assigned Targets: semantic learning Ø Ø … topic model Ø (i-th iteration) (j-th iteration) semantic learning Ø Ø … topic model Ø Inference Stage Ø Ø … semantic learning Ø Ø … … … Figure 1: An example of ONE2SET paradigm at training and inference stages. “Assigned Targets (∗-th iteration)” represents the multiple feasible target permutations gen- erated by K-step target assignment mechanism at dif- ferent training iterations. In this case, both “slot2” and “slot3” are expected to generate keyphrases. However, they often use ∅ token as supervisory signals, and thus over-estimate and output ∅ token. subsequence of the source document.1 Dominant models of keyphrase generation are constructed under three paradigms: ONE2ONE (Meng et al., 2017), ONE2SEQ (Yuan et al., 2020) and ONE2SET (Ye et al., 2021). Among these paradigms, ONE2SET exhibits the state-of-the-art (SOTA) performance. As illustrated in Figure …

Content truncated to 500 words