Linking Databases using Matched Arabic Names

Computational Linguistics and Chinese Language Processing Vol. 19, No. 1, March 2014, pp. 33-54 33  The Association for Computational Linguistics and Chinese Language Processing Linking Databases using Matched Arabic Names Tarek El-Shishtawy Abstract In this paper, a new hybrid algorithm that combines both token-based and character-based approaches is presented. The basic Levenshtein approach also has been extended to the token-based distance metric. The distance metric is enhanced to set the proper granularity level behavior of the algorithm. It smoothly maps a threshold of misspelling differences at the character level and the importance of token level errors in terms of token position and frequency. Using a large Arabic dataset, the experimental results show that the proposed algorithm successfully overcomes many types of errors, such as typographical errors, omission or insertion of middle name components, omission of non-significant popular name, and different writing style character variations. When compared with other classical algorithms, using the same dataset, the proposed algorithm was found to increase the minimum success level of the best tested lower limit algorithm (Soft TFIDF) from 69% to about 80%, while achieving an upper accuracy level of 99.67%. Keywords: Name Matching, Record Linkage, Data Integration, Arabic NLP, Information Retrieval. 1. Introduction Information about individuals can be found in a variety of resources, such as population survey databases, national identifier databases, medical records, news articles, tax information, and educational databases. In all of these heterogeneous sources, name matching is a fundamental task for data integration that joins one or more tables to extract information referring to the same person. Matching personal names has been used in a wide range of applications, such as record linkage or integration, database hardening, removing or cleaning up duplicated records, and searching the Web. Unfortunately, the name may not be known exactly, may be misspelled, or may have spelling variations. Therefore, in these applications, the general word matching  Prof. Ass. of Computer Engineering, Faculty of Computers and Information, Benha University E-mail: t.shishtawy@ictp.edu.eg 34 Tarek El-Shishtawy techniques are insufficient, and optimized techniques have been developed to cope with matching multiple variations of the same personal name. There are efficient and well-established algorithms that deal with spelling errors, variants for strings, and name matching at a character level. For relatively short names that contain similar yet orthographically distinct tokens, character-based measures are preferable because they can estimate the difference between the strings with higher resolution (Bilenko et al., 2003a; Bilenko & Mooney, 2003). In languages where names have very close typographic structure, such as Arabic, character level similarity is not enough to produce high precision matching. Unfortunately, spelling errors are not the only source of name mismatching. People may report their names inconsistently by removing or inserting additional name tokens, adding initial titles, or writing different punctuation marks and whitespaces. In all of these cases, bag-of-words methods are better suited to the matching problem, since they are more flexible at word level. In addition, token-based approaches are not able to capture the degree of similarity between individual …

Content truncated to 500 words