Page Stream Segmentation with Convolutional Neural Nets Combining Textual and Visual Features

Page Stream Segmentation with Convolutional Neural Nets Combining Textual and Visual Features Gregor Wiedemann, Gerhard Heyer Department of Computer Science Leipzig University, Germany gregor.wiedemann@uni-leipzig.de, heyer@informatik.uni-leipzig.de Abstract In recent years, (retro-)digitizing paper-based files became a major undertaking for private and public archives as well as an important task in electronic mailroom applications. As a first step, the workflow involves scanning and Optical Character Recognition (OCR) of documents. Preservation of document contexts of single page scans is a major requirement in this context. To facilitate workflows involving very large amounts of paper scans, page stream segmentation (PSS) is the task to automatically separate a stream of scanned images into multi-page documents. In a digitization project together with a German federal archive, we developed a novel approach based on convolutional neural networks (CNN) combining image and text features to achieve optimal document separation results. Evaluation shows that our PSS architecture achieves an accuracy up to 93 % which can be regarded as a new state-of-the-art for this task. Keywords: page stream segmentation, convolutional neural nets, document image classification, document management, text classification 1. Introduction For digitization of incoming mails in business contexts as well as for retro-digitizing archives, batch scanning of documents can be a major simplification of the process- ing workflow. In this scenario, scanned images of multi- page documents arrive at a document management system as an ordered stream of single pages lacking information on document boundaries. Page stream segmentation (PSS) then is the task of dividing the continuous document stream into sequences of pages that represent single physical doc- uments.1 Applying a fully automated approach of document page segmentation can be favorable over manually separating and scanning documents, especially in contexts of very large data sets which need to be separated (Gallo et al., 2016). In a joint research project together with a German research archive, we supported the task of retro-digitization of a paper archive consisting of circa one million pages put on file between 1922 and 2010 (Isemann et al., 2014). The collection contains documents of varying content, types and lengths around the topic of ultimate disposal of nuclear waste, mostly administrative letter correspondence and re- search reports, but also stock lists, meeting minutes and email printouts. The 1M pages were archived in roughly 20.000 binders which were batch-scanned due to limited manual capacities for separating individual documents. The long time range of archived material affects document qual- ity, proliferation of layout standards, different fonts and the use of hand-written texts. All these circumstances pose se- vere challenges to OCR as well as to page stream segmen- tation (PSS). In this article, we introduce our approach to PSS compar- ing (linear) support vector machines (SVM) and convolu- tional neural networks (CNN). For the first time for this task, we combine textual and visual features into one net- 1The task is also referred to as Document Flow Segmentation or Document Separation. work to achieve most-accurate results. The upcoming sec- tion 2. elaborates on related work. In section 3. we …

Content truncated to 500 words