Combining Visual and Textual Features for Information Extraction from Online Flyers

Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1924–1929, October 25-29, 2014, Doha, Qatar. c⃝2014 Association for Computational Linguistics Combining Visual and Textual Features for Information Extraction from Online Flyers Emilia Apostolova BrokerSavant Inc 2506 N. Clark St. Chicago, IL 60614 emilia@brokersavant.com Noriko Tomuro DePaul University 243 S. Wabash Ave. Chicago, IL 60604 tomuro@cs.depaul.edu Abstract Information in visually rich formats such as PDF and HTML is often conveyed by a combination of textual and visual features. In particular, genres such as marketing flyers and info-graphics often augment textual information by its color, size, positioning, etc. As a result, tradi- tional text-based approaches to informa- tion extraction (IE) could underperform. In this study, we present a supervised ma- chine learning approach to IE from on- line commercial real estate flyers. We evaluated the performance of SVM clas- sifiers on the task of identifying 12 types of named entities using a combination of textual and visual features. Results show that the addition of visual features such as color, size, and positioning significantly increased classifier performance. 1 Introduction Since the Message Understanding Conferences in the 1990s (Grishman and Sundheim, 1996; Chin- chor and Robinson, 1997), Information Extraction (IE) and Named Entity Recognition (NER) ap- proaches have been applied and evaluated on a va- riety of domains and textual genres. The majority of the work, however, focuses on the journalistic, scientific, and informal genres (newswires, scien- tific publications, blogs, tweets, and other social media texts) (Nadeau and Sekine, 2007) and deals with purely textual corpora. As a result, the fea- ture space of NER systems involves purely tex- tual features, typically word attributes and char- acteristics (orthography, morphology, dictionary lookup, etc.), their contexts and document features (surrounding word window, local syntax, docu- ment/corpus word frequencies, etc.) (Nadeau and Sekine, 2007). At the same time, textual information is often presented in visually rich formats, e.g. HTML and PDF. In addition to text, these formats use a vari- ety of visually salient characteristics, (e.g. color, font size, positioning) to either highlight or aug- ment textual information. In some genres and do- mains, a textual representation of the data, exclud- ing visual features is often not enough to accu- rately identify named entities of interest or extract relevant information. Marketing materials, such as online flyers or HTML emails, often contain a plethora of visual features and text-based NER approaches lead to poor results. In this paper, we present a supervised approach that uses a combi- nation of textual and visual features to recognize named entities in online marketing materials. 2 Motivation and Problem Definition A number of broker-based industries (e.g. com- mercial real estate, heavy equipment machinery, etc.) lack a centralized searchable database with industry offerings. In particular, the commercial real estate industry (unlike residential real estate) does not have a centralized database or an estab- lished source of information. Commercial real estate brokers often need to rely on networking, chance, and waste time with a variety of commer- cial real …

Content truncated to 500 words