Monitoring Fact Preservation, Grammatical Consistency and Ethical Behavior of Abstractive Summarization Neural Models

Proceedings of Recent Advances in Natural Language Processing, pages 901–909 Sep 1–3, 2021. https://doi.org/10.26615/978-954-452-072-4_103 901 Monitoring Fact Preservation, Grammatical Consistency and Ethical Behavior of Abstractive Summarization Neural Models Iva Marinova Identrics iva.marinova@identrics.net Yolina Petrova Identrics yolina.petrova@identrics.net Milena Slavcheva IICT, Bulgarian Academy of Sciences milena@lml.bas.bg Petya Osenova Ivaylo Radev IICT, Bulgarian Academy of Sciences {petya, radev, kivs}@bultreebank.org Kiril Simov Abstract The paper describes a system for automatic summarization in English language of on- line news data that come from different non- English languages. The system is designed to be used in production environment for media monitoring. Automatic summarization can be very helpful in this domain when applied as a helper tool for journalists so that they can re- view just the important information from the news channels. However, like every software solution, the automatic summarization needs performance monitoring and assured safe en- vironment for the clients. In media monitoring environment the most problematic features to be addressed are: the copyright issues, the fac- tual consistency, the style of the text and the ethical norms in journalism. Thus, the main contribution of our present work is that the above mentioned characteristics are success- fully monitored in neural automatic summa- rization models and improved with the help of validation, fact-preserving and fact-checking procedures. 1 Introduction Automatic summarization is the task of retelling long texts in a shorter abstract, emphasizing the most important information in an easy to read and grammatically correct way. There are two types of automatic summarization: extractive and abstrac- tive. The key difference between the two are the methods they use to summarize text documents. While the extractive summarization is a scoring task that looks for the most important sentences within the input text, the abstractive summariza- tion is a text generation task, relying on machine- based understanding of the content of the original text. The end-product of the abstractive summa- rization has an element of innovation - it often contains phrases and knowledge which are not part of the source document but can be inferred from it. This creativity element makes most abstractive summaries closer to human-made ones. Thus, we consider abstractive summarization to be more ap- propriate for our main goal - producing a system for automatic summarization in English of textual data that come from different non-English languages, applicable in the production environment for me- dia monitoring. Additionally, it is mandatory in journalism to retell information complying with copyright rules, which is not possible to achieve with extractive summarization. On the other hand, recent research shows that the state-of-the-art (SOTA) abstractive summariza- tion neural models have difficulties with the proper processing of lengthy texts. The problem is that the models’ attention focuses mostly on the beginning of the text source, which means that the information appearing at the end of the text is either truncated (if the input text exceeds the allowed length) or/and is simply ignored (Raffel et al., 2020). As a con- sequence, the generated summary itself does not capture all the relevant information. Another …

Content truncated to 500 words