Enhancing Usability of Information Extraction Results with Textual Data Profiling

Proceedings of PACLIC 19, the 19th Asia-Pacific Conference on Language, Information and Computation. Enhancing Usability of Information Extraction Results with Textual Data Profiling Jyi-Shane Liu and Yung-Wei Cheng Department of Computer Science National Chengchi University 64 Sec. 2 Chihnan Rd., Taipei 116, Taiwan jsliu@cs.nccu.edu.tw Abstract Given a targeted subject and a text collection, information extraction techniques provide the capability to populate a database in which each record entry is a subject instance documented in the text collection. However, even with the state-of-the-art IE techniques, IE task results are expected to contain errors. Manual error detection and correction are labor intensive and time consuming. This validation cost remains a major obstacle to actual deployment of practical IE applications with high validity requirement. In this paper, we propose a string feature-based approach to textual data profiling and invalid data detection. The approach is based on the observation that values of an attribute in IE results are symbolic form variations of a concept in the IE task subject and may exhibit a certain congruity with some string features. We conducted experiments to verify that effective detection of IE invalid values can be achieved by using the surface-form string features. 1. Introduction Most IE researches are concerned with producing better performance and facilitating development portability [2][5][1]. Even with the state-of-the-art IE techniques, IE task results are expected to contain errors. If the IE results are to be used for applications with data validity concerns, IE errors need to be detected and corrected before any value-added processing can be applied. For a typical IE application with tens of thousands, or even hundreds of thousands, extracted entities, manual error detection and correction are labor intensive and time consuming in checking IE results with original documents. This validation cost remains a major obstacle to actual deployment of practical IE applications. In some cases, the validation cost may be so high that IE techniques appear to be hardly superior to direct human work in which both extraction and validation may be performed at the same time. Unfortunately, issues of ensuring highly validated IE results with acceptable cost are little addressed in the IE community. If IE techniques are to be translated into adequately supported actual deployment, we must have some ways to reduce validation cost and ensure data quality of IE results. Data cleansing, also called data cleaning, considers the problem of identifying and removing errors and inconsistencies in data sets [3]. Due to the wide range of possible data errors and inconsistencies, data cleansing involves a variety of research efforts to deal with invalid data, missing data, and duplicated data that occur in single source or when integrating multiple sources. One of the primary focuses is on data integrity analysis or data auditing. The goal is to analyze the actual instances of data to obtain indicative data characteristics or value patterns. The results of data analysis are then used to locate potential errors and inconsistencies based on anomaly and conflict detection. Two related approaches, data profiling and …

Content truncated to 500 words