DeepNLPF: A Framework for Integrating Third Party NLP Tools

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 7244–7251 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 7244 DeepNLPF: A Framework for Integrating Third-Party NLP Tools Francisco Rodrigues1, Rinaldo Lima2, William Domingues3, Robson Fidalgo1, Adrian Chifu3, Bernard Espinasse3, Sébastien Fournier3 1 Centro de Informática, Universidade Federal de Pernambuco, Recife, Brazil 2 Departamento de Computação, Universidade Federal Rural de Pernambuco, Recife, Brazil 3 LIS UMR CNRS 7020, Aix-Marseille Université/Université de Toulon, France rinaldo.jose@ufrpe.br, {fasr, rdnf}@cin.ufpe.br {william.domingues, adrian.chifu, bernard.espinasse, sebastien.fournier}@lis-lab.fr Abstract Natural Language Processing (NLP) of textual data is usually broken down into a sequence of several subtasks, where the output of one the subtasks becomes the input to the following one, which constitutes an NLP pipeline. Many third-party NLP tools are currently available, each performing distinct NLP subtasks. However, it is difficult to integrate several NLP toolkits into a pipeline due to many problems, including different input/output representations or formats, distinct programming languages, and tokenization issues. This paper presents DeepNLPF, a framework that enables easy integration of third-party NLP tools, allowing the user to preprocess natural language texts at lexical, syntactic, and semantic levels. The proposed framework also provides an API for complete pipeline customization including the definition of input/output formats, integration plugin management, transparent multiprocessing execution strategies, corpus-level statistics, and database persistence. Furthermore, the DeepNLPF user-friendly GUI allows its use even by a non-expert NLP user. We conducted runtime performance analysis showing that DeepNLPF not only easily integrates existent NLP toolkits but also reduces significant runtime processing compared to executing the same NLP pipeline in a sequential manner. Keywords: Natural Language Processing, NLP tools integration, Framework 1. Introduction The automatic processing of natural language texts has been increasingly employed in many text mining applications, such as information extraction, automatic summarization, sentiment analysis, etc. Usually this processing is broken down into subtasks where the output of one the subtasks becomes the input to the following one, which constitutes a pipeline. Many freely available Natural Language Processing (NLP) tools for many languages have already been proposed in the literature. Such tools usually provide distinct and even complementary subtasks that would be of great interest if the user could easily integrate them first. However, the use of these tools, their integration according to a specific pipeline may raise many issues including different input and output representation formats, programming languages, tokenization conflicts, to name a few. Moreover, most of the NLP subtasks at a higher level (semantic parsing) usually require a lot of computation resources, thus it is essential to optimize their execution taking advantage of modern CPU-based architectures allowing multiprocessing or parallelism. In this paper we present DeepNLPF, a framework that promotes an easy integration of third-party NLP tools allowing the preprocessing of natural language texts at lexical, syntactic, and semantic levels. The proposed framework also provides an API for complete pipeline customization including the definition of input/output formats, integration plugin management, transparent multiprocessing execution strategies, corpus- level statistics, and database persistence. Furthermore, …

Content truncated to 500 words