Group based Self Training for E-Commerce Product Record Linkage

Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, pages 1311–1321, Dublin, Ireland, August 23-29 2014. Group based Self Training for E-Commerce Product Record Linkage Wayne Xin Zhao1,2, Yuexin Wu2, Hongfei Yan2 and Xiaoming Li2 1School of Information, Renmin University of China, China 2School of Electronic Engineering and Computer Science, Peking University, China batmanfly@gmail.com, wuyuexin@gmail.com, yhf1029@gmail.com, lxm@pku.edu.cn Abstract In this paper, we study the task of product record linkage across multiple e-commerce web- sites. We solve this task via a semi-supervised approach and adopt the self-training algorithm for learning with little labeled data. In previous self-training algorithms, the learner tries to convert the most confidently predicted unlabeled examples of each class into labeled training examples. However, they evaluate the confidence of an instance only based on the individual evidence from the instance. The correlation among data instances is rarely considered. To address it, we develop a novel variant of the self-training algorithm by leveraging the data characteristics for the task of product record linkage. We joint consider a candidate linked pair and its corresponding correlated pairs as a group at the selection of pseudo labeled data. We propose a novel confidence evaluation method for a group of instances, and incorporate it as a re-ranking step in the self-training algorithm. We evaluate the novel self-training algorithm on two large datasets constructed based on real e-commerce Websites. We adopt several competitive methods as comparisons and perform extensive experiments. The results show that our method outperforms these baselines that do not consider data correlation. 1 Introduction Recent years have witnessed the rapid development of online e-commerce business, e.g. Amazon and eBay, which raises the need for better storing, organizing and analyzing the large amount of product records. An important task is how to effectively link product records across multiple databases or web- sites. This task serves as a fundamental step for many applications. For example, it will be useful to provide entity-oriented search and product comparison analysis in eBay, where record linkage can help to unify the corresponding records (i.e. records from different sellers) given a product. Record linkage has been shown to be important in many fields, including biology (Needleman and Wunsch, 1970), database (Neiling, 2006) and text mining (Goiser and Christen, 2006; Bilenko and Mooney, 2003). In this paper, we mainly focus on the task of product record linkage for online e-commerce websites, but our method is easy to be extended to other data sources and tasks. Early studies on record linkage were mainly based on the classical probabilistic approach develope- d by Fellegi and Sunter (1969), furthermore it was improved by the application of the expectation- maximization (EM) algorithm (Winkler, 1988) and the use of approximate string comparison algorithms (Christen, 2006; Winkler, 2006). The early work was not flexible to incorporate rich information. The development of machine learning techniques in the late 1990s provides a new approach for record link- age, and it has become the mainstream methodology for this task. The task of record linkage …

Content truncated to 500 words