Boosting Deep CTR Prediction with a Plug-and-Play Pre-trainer for News Recommendation

Proceedings of the 29th International Conference on Computational Linguistics, pages 2823–2833 October 12–17, 2022. 2823 Boosting Deep CTR Prediction with a Plug-and-Play Pre-trainer for News Recommendation Qijiong Liu The Hong Kong Polytechnic University jyonn.liu@connect.polyu.hk Jieming Zhu Huawei Noah’s Ark Lab jiemingzhu@ieee.org Quanyu Dai Huawei Noah’s Ark Lab quanyu.dai@connect.polyu.hk Xiaoming Wu∗ The Hong Kong Polytechnic University xiao-ming.wu@polyu.edu.hk Abstract Understanding textual content is critical to im- proving the quality of news recommendation. To achieve this goal, recent studies have pro- posed to apply pre-trained language models (PLMs) such as BERT for semantic-enhanced news recommendation. Despite their great suc- cess in offline evaluation, it is challenging to apply such large PLMs in real-time ranking tasks due to the stringent latency requirements in model updating and inference. To bridge this gap, we propose a plug-and-play pre-trainer, namely PREC, to learn both user and news encoders through multi-task pre-training. In- stead of directly leveraging sophisticated PLMs for end-to-end inference, we focus on how to use the cached user and item representations to boost the performance of traditional ID-based models for click-through-rate prediction. This enables efficient online inference as well as compatibility to the widely-used models in in- dustry, which would significantly ease the prac- tical deployment. We validate the effectiveness of PREC through both offline evaluation on public datasets and online A/B testing in an industrial system. 1 Introduction Personalized news recommendation has become a ubiquitous channel in various online applications, such as Google News and MSN News, which helps users discover their interested news information. To deal with the massive amounts (usually mil- lions) of daily news, industrial recommender sys- tems usually apply a multi-stage recommendation pipeline as illustrated in Figure 1. It generally in- volves two phases, matching and ranking (Coving- ton et al., 2016). The matching phase first generates hundreds or thousands of news candidates from multiple channels (e.g., popularity-based channel, content-based channel (Wu et al., 2019a,b), and collaborative filtering channel (Linden et al., 2003; ∗ Corresponding author. Figure 1: A multi-stage pipeline for news recommen- dation. Sedhain et al., 2014)). Subsequently, the ranking phase employs a news ranking task, such as click- through rate (CTR) prediction, to rank the candi- date news and finally return the top dozens of news to the user interface. The matching and ranking phases have different goals and requirements. Concretely, the match- ing phase aims for efficient and high-recall candi- date retrieval from millions of news corpus. There- fore, most studies construct two-tower networks (e.g., NPA (An et al., 2019), LSTUR (Wu et al., 2019a), and NRMS (Wu et al., 2019b)) to learn user representations and item representations sepa- rately, and then apply simple dot product to com- pute similarity scores. As such, both user and item representations can be pre-computed offline and cached online for fast nearest neighbour search (e.g., using Faiss (Johnson et al., 2019)). In con- trast, the ranking phase targets at accurate scor- ing of each candidate item based on CTR predic- tion and thus requires networks to capture com- plex feature …

Content truncated to 500 words