Retrieval Term Prediction Using Deep Belief Networks

PACLIC 28 !338 Retrieval Term Prediction Using Deep Belief Networks Qing Ma† Ibuki Tanigawa† Masaki Murata‡ † Department of Applied Mathematics and Informatics, Ryukoku University ‡ Department of Information and Electronics, Tottori University qma@math.ryukoku.ac.jp Abstract This paper presents a method to predict re- trieval terms from relevant/surrounding words or descriptive texts in Japanese by using deep belief networks (DBN), one of two typical types of deep learning. To determine the effec- tiveness of using DBN for this task, we tested it along with baseline methods using example- based approaches and conventional machine learning methods, i.e., multi-layer perceptron (MLP) and support vector machines (SVM), for comparison. The data for training and test- ing were obtained from the Web in manual and automatic manners. Automatically cre- ated pseudo data was also used. A grid search was adopted for obtaining the optimal hyper- parameters of these machine learning meth- ods by performing cross-validation on training data. Experimental results showed that (1) us- ing DBN has far higher prediction precisions than using baseline methods and higher pre- diction precisions than using either MLP or SVM; (2) adding automatically gathered data and pseudo data to the manually gathered data as training data is an effective measure for fur- ther improving the prediction precisions; and (3) DBN is able to deal with noisier training data than MLP, i.e., the prediction precision of DBN can be improved by adding noisy train- ing data, but that of MLP cannot be. 1 Introduction The current Web search engines have a very high retrieval performance as long as the proper retrieval terms are given. However, many people, particularly children, seniors, and foreigners, have difficulty de- ciding on the proper retrieval terms for represent- ing the retrieval objects,1 especially with searches related to technical fields. The support systems are in place for search engine users that show suitable retrieval term candidates when some clues such as their descriptive texts or relevant/surrounding words are given by the users. For example, when the relevant/surrounding words “computer”, “previous state”, and “return” are given by users, “system re- store” is predicted by the systems as a retrieval term candidate. Our objective is to develop various domain- specific information retrieval support systems that can predict suitable retrieval terms from rele- vant/surrounding words or descriptive texts in Japanese. To our knowledge, no such studies have been done so far in Japanese. As the first step, here, we confined the retrieval terms to the computer- related field and proposed a method to predict them using machine learning methods with deep belief networks (DBN), one of two typical types of deep learning. In recent years, deep learning/neural network techniques have attracted a great deal of attention in various fields and have been successfully applied not only in speech recognition (Li et al., 2013) and image recognition (Krizhevsky et al., 2012) tasks but also in NLP tasks including morphology & syn- 1For example, according to a questionnaire admin- istered by Microsoft in 2010, about 60% of users had difficulty deciding …

Content truncated to 500 words