Towards High-Quality Text Stream Extraction from PDF. Technical Background to the ACL 2012 Contributed Task

Proceedings of the ACL-2012 Special Workshop on Rediscovering 50 Years of Discoveries, pages 98–103, Jeju, Republic of Korea, 10 July 2012. c⃝2012 Association for Computational Linguistics Towards High-Quality Text Stream Extraction from PDF Technical Background to the ACL 2012 Contributed Task Øyvind Raddum Berg, Stephan Oepen, and Jonathon Read Department of Informatics, Universitetet i Oslo {oyvinrb|oe|jread}@ifi.uio.no Abstract Extracting textual content and document structure from PDF presents a surprisingly (depressingly, to some, in fact) difficult chal- lenge, owing to the purely display-oriented de- sign of the PDF document standard. While a variety of lower-level PDF extraction toolk- its exist, none fully support the recovery of original text (in reading order) and relevant structural elements, even for so-called born- digital PDFs, i.e. those prepared electronically using typesetting systems like LATEX, OpenOf- fice, and the like. This short paper summarizes a new tool for high-quality extraction of text and structure from PDFs, combining state-of- the-art PDF parsing, font interpretation, layout analysis, and TEI-compliant output of text and logical document markup.† 1 Introduction—Motivation To view a collection of scholarly articles like the ACL Anthology as a structured knowledge base sub- stantially transcends a naïve notion of a corpus as a mere collection of running text. Research litera- ture is the result of careful editing and typesetting and, thus, is organized around its complex internal structure. Relevant structural elements can comprise both geometric (e.g. pages, columns, blocks, or ta- bles) and logical units (e.g. titles, abstracts, head- ings, paragraphs, or citations)—where (ideally) ge- ometric and logical document structure play hand in hand to a degree that can make it hard to draw clear dividing lines in some cases (e.g. in itemized or numbered lists). To date, the dominant standard for electronic doc- ument archival is Portable Document Format (PDF), †We are indebted to Rebecca Dridan, Ulrich Schäfer, and the ACL workshop reviewers for helpful feedback on this work. originally created as a proprietary format by Adobe Systems Incorporated in the early 1990s and sub- sequently made an open ISO standard (which was officially adopted in 2008 and embraced by Adobe through a public license that grants royalty-free us- age). PDF is something of a composite standard, unifying at least three basic technologies: 1. A subset of the PostScript page ‘programming’ language, dropping constructs like loops and branches, but including all graphical operations to draw layout elements, text, and images. 2. A font embedding system which allows a doc- ument to ‘carry along’ a broad variety of fonts (in various formats), as may be needed to en- sure display just as the document was designed. 3. A structured storage system, which organizes various data objects—for example images and fonts—inside a PDF document. All data objects in a PDF file are represented in a visually-oriented way, as a sequence of operators which—when interpreted by a PDF renderer—will draw the document on a page canvas. This is a nat- ural approach considering the design roots of PDF as a PostScript successor and its original central role in desktop …

Content truncated to 500 words