FPC: Fine-tuning with Prompt Curriculum for Relation Extraction

Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 1065–1077 November 20–23, 2022. ©2022 Association for Computational Linguistics 1065 FPC: Fine-tuning with Prompt Curriculum for Relation Extraction Sicheng Yang, Dandan Song∗ School of Computer Science and Technology, Southeast Academy of Information Technology, Beijing Engineering Research Center of High Volume Language Information Processing and Cloud Computing Applications, Beijing Institute of Technology, Beijing, China yangsicheng@bit.edu.cn, sdd@bit.edu.cn Abstract The current classification methods for relation extraction (RE) generally utilize pre-trained language models (PLMs) and have achieved superior results. However, such methods di- rectly treat relation labels as class numbers, therefore they ignore the semantics of relation labels. Recently, prompt-based fine-tuning has been proposed and attracted much attention. This kind of methods insert templates into the input and convert the classification task to a (masked) language modeling problem. With this inspiration, we propose a novel method Fine-tuning with Prompt Curriculum (FPC) for RE, with two distinctive characteristics: the relation prompt learning, introducing an auxil- iary prompt-based fine-tuning task to make the model capture the semantics of relation labels; the prompt learning curriculum, a fine-tuning procedure including an increasingly difficult task to adapt the model to the difficult multi- task setting. We have conducted extensive ex- periments on four widely used RE benchmarks under fully supervised and low-resource set- tings. The experimental results show that FPC can significantly outperform the existing meth- ods and obtain the new state-of-the-art results. 1 Introduction As one of the essential tasks in natural language processing (NLP), relation extraction (RE) intends to extract relational facts hidden in text. Figure 1 shows the typical RE setting: a sentence with two marked entities ("Tesla" and "Elon Musk") is input into a model to classify the relation (founded by) be- tween the entities. Structured knowledge captured by RE can benefit many downstream applications such as knowledge graph completion (Bordes et al., 2013), dialogue systems (Madotto et al., 2018) and question answering (Bordes et al., 2014). As the mainstream of RE, the classification meth- ods extract semantic features from text to form ∗Corresponding author Sentence: Elon Musk is known for co-founding Tesla . founded by Figure 1: An example to show the typical RE setting. relation representations (vectors). Then the repre- sentations are fed into a classifier to predict relation labels. The recent classification methods generally utilize pre-trained language models (PLMs) and have achieved promising results. This is because self-supervised learning on large-scale unlabeled data makes PLMs obtain rich knowledge, which is important for natural language understanding (De- vlin et al., 2019; Liu et al., 2019) and generation (Raffel et al., 2020; Lewis et al., 2020). However, such methods directly treat relation labels as class numbers, hence they can not capture the semantics of relation labels. On the contrary, the reformulation methods can improve the deficiency by intuitively transform RE into other tasks such as question answering (QA) (Levy et al., 2017). For example, …

Content truncated to 500 words