Mining Tables from Large Scale HTML Texts

Mining Tables from Large Scale HTML Texts Hsin-Hsi Chen, Shih-Chung Tsai and Jin-He Tsai Department of Computer Science and Information Engineering National Taiwan University Taipei, TAIWAN, R.O.C. E-mail: hh_chen@csie.ntu.edu.tw Abstract Table is a very common presentation scheme, but few papers touch on table extraction in text data mining. This paper focuses on mining tables from large-scale HTML texts. Table filtering, recognition, interpretation, and presentation are discussed. Heuristic rules and cell similarities are employed to identify tables. The F-measure of table recognition is 86.50%. We also propose an algorithm to capture attribute-value relationships among table cells. Finally, more structured data is extracted and presented. Introduction Tables, which are simple and easy to use, are very common presentation scheme for writers to describe schedules, organize statistical data, summarize experimental results, and so on, in texts of different domains. Because tables provide rich information, table acquisition is useful for many applications such as document understanding, question-and-answering, text retrieval, etc. However, most of previous approaches on text data mining focus on text parts, and only few touch on tabular ones (Appelt and Israel, 1997; Gaizauskas and Wilks, 1998; Hurst, 1999a). Of the papers on table extractions (Douglas, Hurst and Quinn, 1995; Douglas and Hurst 1996; Hurst and Douglas, 1997; Ng, Lim and Koo, 1999), plain texts are their targets. In plain text, writers often use special symbols, e.g., tabs, blanks, dashes, etc., to make tables. The following shows an example. It depicts book titles, authors, and prices. title author price Statistical Language Learning E.Charniak $30 Cross-Language Information Retrieval G. Grefenstette $115 Natural Language Information Retrieval T.Strzalkowski $144 When detecting if there is a table in free text, we should disambiguate the uses of the special symbols. That is, the special symbol may be a separator or content of cells. Previous papers employ grammars (Green and Krishnamoorthy, 1995), string-based cohesion measures (Hurst and Douglas, 1997), and learning methods (Ng, Lim and Koo, 1999) to deal with table recognition. Because of the simplicity of table construction methods in free text, the expressive capability is limited. Comparatively, the markup languages like HTML provide very flexible constructs for writers to design tables. The flexibility also shows that table extraction in HTML texts is harder than that in plain text. Because the HTML texts are huge on the web, and they are important sources of knowledge, it is indispensable to deal with table mining on HTML texts. Hurst (1999b) is the first attempt to collect a corpus from HTML files, LATEX files and a small number of ASCII files for table extraction. This paper focuses on HTML texts. We will discuss not only how to recognize tables from HTML texts, but also how to identify the roles of each cell (attribute and/or value), and how to utilize the extracted tables. 1 Tables in HTML HTML table begins with an optional caption followed one or more rows. Each row is formed by one or more cells, which are classified into header and data cells. Cells can be merged across rows …

Content truncated to 500 words