Learning Robust Models for e-Commerce Product Search

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6861–6869 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 6861 Learning Robust Models for e-Commerce Product Search Thanh V. Nguyen Iowa State University thanhng@iastate.edu Nikhil Rao Amazon nikhilsr@amazon.com Karthik Subbian Amazon ksubbian@amazon.com Abstract Showing items that do not match search query intent degrades customer experience in e- commerce. These mismatches result from counterfactual biases of the ranking algorithms toward noisy behavioral signals such as clicks and purchases in the search logs. Mitigating the problem requires a large labeled dataset, which is expensive and time-consuming to ob- tain. In this paper, we develop a deep, end- to-end model that learns to effectively classify mismatches and to generate hard mismatched examples to improve the classifier. We train the model end-to-end by introducing a latent variable into the cross-entropy loss that alter- nates between using the real and generated samples. This not only makes the classifier more robust but also boosts the overall rank- ing performance. Our model achieves a rela- tive gain compared to baselines by over 26% in F-score, and over 17% in Area Under PR curve. On live search traffic, our model gains significant improvement in multiple countries. 1 Introduction Deep learning models have shown excellent per- formance in the natural language domain, and this success has inspired practitioners to adapt these models to information retrieval tasks (Mitra et al., 2017; Huang et al., 2013). However, deep learn- ing has not succeeded in these tasks due to the lack of massive labeled datasets (Dehghani et al., 2017). Another reason is that word-based repre- sentations (Mikolov et al., 2013; Pennington et al., 2014) are less useful in representing complex, infor- mal search queries (Xiong et al., 2017) and hence provide limited understanding of the search intent. In the absence of explicit knowledge of which docu- ments are “matched” with a search query and which are “mismatched”, it is hard to learn robust deep learning models that understand the query intent and find high-quality, relevant documents. Text-based product search is even more challeng- ing. Simple modifications to the input query (or a product title) can completely change the search intent (or the product type, respectively). Take, for example, the query gray iPhone X by which a user is looking for a specific phone. Slightly mod- ified queries such as iPhone X charger and case for iPhone X refer to different prod- ucts. Therefore, it is hard for distributed represen- tations to capture the nuances. Moreover, noisy user-behavioral signals from clicks and purchases (e.g., users purchased a phone while searching for a charger) can lead to biases in the ranking algo- rithms. As such, even top-ranked items may not match the search intent. In this paper, we consider the problem of identi- fying query-item mismatches to enhance the rank- ing performance in product search. This task typ- ically requires a large labeled dataset of matches and mismatches that we will respectively refer to as negative and positive samples. …

Content truncated to 500 words