RecoBERT: A Catalog Language Model for Text-Based Recommendations

Findings of the Association for Computational Linguistics: EMNLP 2020, pages 1704–1714 November 16 - 20, 2020. c⃝2020 Association for Computational Linguistics 1704 RecoBERT: A Catalog Language Model for Text-Based Recommendations Itzik Malkiel1,2,Oren Barkan1,3 ,Avi Caciularu1,4 ,Noam Razin1,2 ,Ori Katz1,5 and Noam Koenigstein1,2 1Microsoft 2Tel Aviv University 3Ariel University 4Bar-Ilan University 5Technion {itmalkie, orenb, Ori.Katz, Noam.Koenigstein}@microsoft.com avi.c33@gmail.com noam.razin@cs.tau.ac.il Abstract Language models that utilize extensive self- supervised pre-training from unlabeled text, have recently shown to significantly advance the state-of-the-art performance in a variety of language understanding tasks. However, it is yet unclear if and how these recent mod- els can be harnessed for conducting text-based recommendations. In this work, we introduce RecoBERT, a BERT-based approach for learn- ing catalog-specialized language models for text-based item recommendations. We sug- gest novel training and inference procedures for scoring similarities between pairs of items, that don’t require item similarity labels. Both the training and the inference techniques were designed to utilize the unlabeled structure of textual catalogs, and minimize the discrep- ancy between them. By incorporating four scores during inference, RecoBERT can infer text-based item-to-item similarities more ac- curately than other techniques. In addition, we introduce a new language understanding task for wine recommendations using similar- ities based on professional wine reviews. As an additional contribution, we publish anno- tated recommendations dataset crafted by hu- man wine experts. Finally, we evaluate Re- coBERT and compare it to various state-of-the- art NLP models on wine and fashion recom- mendations tasks. 1 Introduction Recommendation systems are a major component of content discovery in online stores. Different recommendation systems are employed across a broad spectrum of domains, such as movies, music, groceries, and more. In each case, the recommenda- tion system is associated with a different catalog of items comprising different descriptors, item proper- ties, and metadata. This work deals with the case of generating item-to-item similarities based on item descriptions. Personalized recommender systems make use of either or both Collaborative Filtering (CF) or Content-Based (CB) information (Aggarwal et al., 2016). CF approaches build models based on users past behavior (Breese et al., 2013; Schafer et al., 2007). On the other hand, CB recommenders use item meta-data such as properties, tags, and de- scriptions in order to build and match user and item profiles (Brusilovski et al., 2007; Wang et al., 2018c; Lops et al., 2011). A model that utilizes both CF and CB is called a hybrid recommender system. Item-to-item recommendations are commonly used in large scale recommender systems such as on Netflix (Gomez-Uribe and Hunt, 2015), Amazon (Linden et al., 2003), Xbox (Koenigstein and Paquet, 2013) and many others. Commonly found on product details page (PDP), these non- personalized recommendation lists are known to drive-up purchases as well as user engagement. Similar to personalized recommendation, item sim- ilarities can be computed based on user activity, item meta-data or both, using a variety of differ- ent models. In a new store, where user data does not exist, item-to-item recommendations are com- puted using one or more content-based approaches …

Content truncated to 500 words