Learning to Recognize Tables in Free Text

Learning to Recognize Tables in Free Text Hwee Tou Ng Chung Yong Lim Jessica Li Teng Koo DSO National Laboratories 20 Science Park Drive, Singapore 118230 {nhweetou, ichungyo, kliteng}@dso, org. sg Abstract Many real-world texts contain tables. In order to process these texts correctly and extract the infor- mation contained within the tables, it is important to identify the presence and structure of tables. In this paper, we present a new approach that learns to recognize tables in free text, including the bound- ary, rows and columns of tables. When tested on Wall Street Journal news documents, our learning approach outperforms a deterministic table recogni- tion algorithm that identifies tables based on a fixed set of conditions. Our learning approach is also more flexible and easily adaptable to texts in different do- mains with different table characteristics. 1 Introduction Tables are present in many reai-world texts. Some information such as statistical data is bestpresented in tabular form. A check on the more than 100,000 Wall Street Journal (WSJ) documents collected in the ACL/DCI CD-ROM reveals that at least an es- timated one in 30 documents contains tables. Tables present a unique challenge to information extraction systems. At the very least, the presence of tables must be detected so that they can be skipped over. Otherwise, processing the lines that consti- tute tables as if they are normal "sentences" is at best misleading and at worst may lead to erroneous analysis of the text. As tables contain important data and information, it is critical for an information extraction system to be able to extract the information embodied in ta- bles. This can be accomplished only if the structure of a table, including its rows and columns, are iden- tified. That table recognition is an important step in in- formation extraction has been recognized in (Appelt and Israel, 1997). Recently, there is also a greater realization within the computational linguistics com- munity that the layout and types of information (such as tables) contained in a document are im- portant considerations in text processing (see the call for participation (Power and Scott, 1999) for the 1999 AAAI Fail Symposium Series). However, despite the omnipresence of tables and their importance, there is surprisingly very little work in computational linguistics on algorithms to recognize tables. The only research that we are aware of is the work of (Hurst and Douglas, 1997; Douglas and Hurst, 1996; Douglas et al., 1995). Their method is essentially a deterministic algorithm that relies on spaces and special punctuation sym- bols to identify the presence and structure of tables. However, tables are notoriously idiosyncratic. The main difficulty in table recognition is that there axe so many different and varied ways in which tables can show up in real-world texts. Any deterministic algorithm based on a fixed set of conditions is bound to fail on tables with unforeseen layout and structure in some domains. In contrast, we present a new approach in this pa- per that learns to recognize tables in free …

Content truncated to 500 words