Weakly-Supervised Questions for Zero-Shot Relation Extraction

Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 3075–3087 May 2-6, 2023 ©2023 Association for Computational Linguistics Weakly-Supervised Questions for Zero-Shot Relation Extraction Saeed Najafi and Alona Fyshe Department of Computing Science, University of Alberta, Canada {snajafi,alona}@ualberta.ca Abstract Zero-Shot Relation Extraction (ZRE) is the task of Relation Extraction where the train- ing and test sets have no shared relation types. This very challenging domain is a good test of a model’s ability to generalize. Previous ap- proaches to ZRE reframed relation extraction as Question Answering (QA), allowing for the use of pre-trained QA models. However, this method required manually creating gold ques- tion templates for each new relation. Here, we do away with these gold templates and instead learn a model that can generate questions for unseen relations. Our technique can success- fully translate relation descriptions into relevant questions, which are then leveraged to generate the correct tail entity. On tail entity extraction, we outperform the previous state-of-the-art by more than 16 F1 points without using gold ques- tion templates. On the RE-QA dataset where no previous baseline for relation extraction exists, our proposed algorithm comes within 0.7 F1 points of a system that uses gold question tem- plates. Our model also outperforms the state- of-the-art ZRE baselines on the FewRel and WikiZSL datasets, showing that QA models no longer need template questions to match the performance of models specifically tailored to the ZRE task. Our implementation is available at https://github.com/fyshelab/QA-ZRE. 1 Introduction Building models that capture abstract knowledge rather than just memorizing data is one of the in- spirations for zero-shot benchmarks (Alex et al., 2021). In order to test a model’s ability to extract higher-level knowledge, these benchmarks measure performance on data unlike what is seen during training. For example, in Zero-shot Relation Ex- traction (ZRE), a model trained to extract the name of a company’s CEO should be able to extract the name of a country’s political leader at test time. In ZRE, the test relations do not appear in the training data, so one cannot apply typical relation classification approaches. One method for ZRE is to reframe the task as a Question-Answering (QA) problem by manually creating question templates for each relation type. Extracting the tail entity is accomplished by finding the answer span for the corresponding question template (Levy et al., 2017). However, this method requires gold ques- tion templates ahead of time. Here we ask, is it possible to leverage QA systems without annotat- ing relations with gold question templates? We treat ZRE as a tail entity generation task for which we consider the dual training of ques- tion and answer generators. We create question and answer generators by pre-training the publicly available T5 models (Raffel et al., 2020) on QA corpora, and then fine-tune for ZRE within our proposed learning framework. We investigate four training objectives based on Marginal Maximum Likelihood (MML) optimization, and suggest an off-policy sampling technique to avoid ungrammat- ical spurious questions …

Content truncated to 500 words