A Comparative Study on Abstractive and Extractive Approaches in Summarization of European Legislation Documents

Proceedings of Recent Advances in Natural Language Processing, pages 1645–1651 Sep 1–3, 2021. https://doi.org/10.26615/978-954-452-072-4_184 1645 A Comparative Study on Abstractive and Extractive Approaches In Summarization of European Legislation Documents Valentin Zmiycharov FMI, Sofia University ”St. Kliment Ohridski” Sofia, Bulgaria valentin.zmiycharov @gmail.com Milen Chechev FMI, Sofia University ”St. Kliment Ohridski” Sofia, Bulgaria milen.chechev@fmi.uni-sofia.bg Gergana Lazarova FMI, Sofia University ”St. Kliment Ohridski” Sofia, Bulgaria gerganal@fmi.uni-sofia.bg Todor Tsonkov FMI, Sofia University ”St. Kliment Ohridski” Sofia, Bulgaria ttsonkov@gmail.com Ivan Koychev FMI, Sofia University ”St. Kliment Ohridski” Sofia, Bulgaria koychev@fmi.uni-sofia.bg Abstract Extracting the most important part of legisla- tion documents has great business value be- cause the texts are usually very long and hard to understand. The aim of this article is to eval- uate different algorithms for text summariza- tion on EU legislation documents. The content contains domain-specific words. We collected a text summarization dataset of EU legal docu- ments consisting of 1563 documents, in which the mean length of summaries is 424 words. Experiments were conducted with different al- gorithms using the new dataset. A simple ex- tractive algorithm was selected as a baseline. Advanced extractive algorithms, which use en- coders show better results than baseline. The best result measured by ROUGE scores was achieved by a fine-tuned abstractive T5 model, which was adapted to work with long texts. 1 Introduction Automatic summarization of legislation documents is a rather challenging task, because they usually are very long and hard to understand. Therefore, any progress on this task might have great value for many businesses. Most of the existing methods for text summa- rization are designed for a relatively short text (such as news, web pages, etc.). Most of the avail- able datasets consist of short summaries of articles, news, etc. in which the expected summary is a few sentences long. However, in the case of legal doc- uments both the text and the summary are longer. Dernoncourt et al. (2018) provide a comprehensive overview of the current datasets for summariza- tion, including CNN/Daily Mail (Hermann et al., 2015), Gigaword (Graff and Cieri, 2003), LCSTS (Hu et al., 2015) and others. Noticeably, most of the larger scale summarization datasets consist of relatively short documents. The paper’s aim is to research the applicability of different algorithms for text summarization on a new dataset, called EU legislation documents. The collected documents were parsed, cleaned and prepared to be used for data summarization training by defining pairs of full text and corresponding summary. There are two main approaches for text sum- marization: extractive and abstractive. Extractive summarization means identifying important parts of the text and concatenating them verbatim to pro- duce a summary which is a subset of the sentences from the original text. Abstractive summarization aims to make algorithms that are able to ”under- stand” the whole text and to generate a new shorter text that conveys the most important information from the original one (Sciforce, 2019). The algorithm, which was selected as baseline, uses a classical approach for generating extrac- tive summaries - sentence importance …

Content truncated to 500 words