One2Set: Generating Diverse Keyphrases as a Set

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 4598–4608 August 1–6, 2021. ©2021 Association for Computational Linguistics 4598 ONE2SET: Generating Diverse Keyphrases as a Set Jiacheng Ye1, Tao Gui2∗, Yichao Luo1, Yige Xu1 and Qi Zhang1∗ 1School of Computer Science, Fudan University 2Institute of Modern Languages and Linguistics, Fudan University {yejc19, tgui16, ycluo18, ygxu18, qz}@fudan.edu.cn Abstract Recently, the sequence-to-sequence models have made remarkable progress on the task of keyphrase generation (KG) by concatenating multiple keyphrases in a predefined order as a target sequence during training. However, the keyphrases are inherently an unordered set rather than an ordered sequence. Imposing a predefined order will introduce wrong bias during training, which can highly penalize shifts in the order between keyphrases. In this work, we propose a new training paradigm ONE2SET without predefining an order to concatenate the keyphrases. To fit this paradigm, we propose a novel model that utilizes a fixed set of learned control codes as conditions to generate a set of keyphrases in parallel. To solve the problem that there is no correspondence between each prediction and target during training, we propose a K-step target assignment mechanism via bipartite matching, which greatly increases the diversity and reduces the duplication ratio of generated keyphrases. The experimental results on multiple benchmarks demonstrate that our approach significantly outperforms the state-of-the-art methods. 1 Introduction Keyphrase generation (KG) aims to generate of a set of keyphrases that expresses the high-level se- mantic meaning of a document. These keyphrases can be further categorized into present keyphrases that appear in the document and absent keyphrases that do not. Meng et al. (2017) proposed a sequence-to-sequence (Seq2Seq) model with a copy mechanism (Gu et al., 2016) to predict both present and absent keyphrases. However, the model needs beam search during inference to overgenerate multiple keyphrases, which cannot determine the dynamic number of keyphrases. To ∗ Corresponding authors. deep learning <sep> topic model <eos> topic model <sep> deep learning <eos> deep learning <eos> topic model <eos> topic model <eos> deep learning <eos> (a) (b) ONE2SEQ ONE2SET Figure 1: An example of ground-truth keyphrases (upper) and predictions (lower) under ONE2SEQ and ONE2SET training paradigm. For the ONE2SEQ training paradigm, although the predictions are correct in each keyphrase, they will still be considered wrong due to the shift in keyphrase order, and the model will receive a large penalty. address this, Yuan et al. (2020) proposed the ONE2SEQ training paradigm where each source text corresponds to a sequence of keyphrases that are concatenated with a delimiter ⟨sep⟩ and a terminator ⟨eos⟩. As keyphrases must be ordered before being concatenated, Yuan et al. (2020) sorted the present keyphrases by their order of the first occurrence in the source text and appended the absent keyphrases to the end. During inference, the decoding process terminates when generating ⟨eos⟩, and the final keyphrase predictions are obtained after splitting the sequence by ⟨sep⟩. Thus, a model trained with ONE2SEQ paradigm can …

Content truncated to 500 words