DeepREF: A Framework for Optimized Deep Learning-based Relation Classification

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 4513–4522 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 4513 DeepREF: A Framework for Optimized Deep Learning-based Relation Classification Igor Nascimento1, Rinaldo Lima1, Adrian Chifu2, Bernard Espinasse2 , S´ebastien Fournier2 1 Universidade Federal Rural de Pernambuco (UFRPE), Recife, Brazil 2 Aix-Marseille Universit´e, Universit´e de Toulon, CNRS, LIS UMR 7020, Marseille, France {igor.lucena;rinaldo.jose}@ufrpe.br & {adrian.chifu; bernard.espinasse; sebastien.fournier}@lis-lab.fr Abstract Relation Extraction (RE) is an important basic Natural Language Processing (NLP) task for many applications, including search engines and question-answering systems. There are many studies in this subarea of NLP that continue to be explored, such as the ones concerned by SemEval shared tasks. For many years, several RE systems based on statistical models have been proposed, as well as the frameworks to develop them. We focus on frameworks allowing to develop such RE systems using deep learning models. Such frameworks make it possible to reproduce experiments using many deep learning models and preprocessing techniques. Currently, there are very few frameworks of this type. In this paper, we propose an open and optimizable framework called DeepREF, inspired by two other existing frameworks: OpenNRE and REflex. DeepREF allows the rapid development of deep learning models for Relation Classification (RC). In addition, it enables hyperparameter optimization, and the application of many preprocessing techniques on the input textual data. DeepREF provides means to boost the process of running deep learning models for RC tasks on different datasets and models. DeepREF is evaluated on three reference corpora and has demonstrated competitive results compared to other state-of-the-art RC systems. Keywords: Relation Classification, Framework, DeepREF, Embeddings, SemEval, DDI, Optuna, NLP 1. Introduction RE is a subfield of study in Information Extraction (IE). The main goal of an IE system is to extract specific pieces of information from some document repository, providing structured information as output. The infor- mation extracted from data is basically of three types: named entities, relations, and events. RE has many applications, including document summarization, ma- chine translation, and the automatic construction of thesauri or semantic networks (Pawar et al., 2017). There are many initiatives that tackle the RE chal- lenge, but it is difficult to reproduce them due to the lack of implementation details in the papers introduc- ing them. Such difficulties include the preprocessing method used, the model hyperparameters, and the de- tails of the model implementation. Not all papers pro- vide the implementation code and, even when it is the case, it is still difficult to reproduce these experiments on other datasets, in most of the time. In order to develop and evaluate RE systems efficiently, dedicated RE Frameworks have been proposed. These RE Frameworks allow performing several experiments on many datasets. They provide such a level of mod- ularity and extensibility that makes the preprocessing stage, with different datasets, easier to implement. This paper focuses on frameworks for developing RE systems using deep learning models. Currently there are very few frameworks of this type. …

Content truncated to 500 words