Distantly Supervised Transformers For E-Commerce Product QA

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4008–4017 June 6–11, 2021. ©2021 Association for Computational Linguistics 4008 Distantly Supervised Transformers For E-Commerce Product QA Happy Mittal India Machine Learning Amazon mithappy@amazon.com Aniket Chakrabarti India Machine Learning Amazon chakanik@amazon.com Belhassen Bayar∗ Digital Video Amazon bayarb@amazon.com Animesh Anant Sharma Community Shopping Amazon shanimes@amazon.com Nikhil Rasiwasia India Machine Learning Amazon rasiwasi@amazon.com Abstract We propose a practical instant question an- swering (QA) system on product pages of e- commerce services, where for each user query, relevant community question answer (CQA) pairs are retrieved. User queries and CQA pairs differ significantly in language charac- teristics making relevance learning difficult. Our proposed transformer-based model learns a robust relevance function by jointly learning unified syntactic and semantic representations without the need for human labeled data. This is achieved by distantly supervising our model by distilling from predictions of a syntactic matching system on user queries and simul- taneously training with CQA pairs. Training with CQA pairs helps our model learning se- mantic QA relevance and distant supervision enables learning of syntactic features as well as the nuances of user querying language. Ad- ditionally, our model encodes queries and can- didate responses independently allowing of- fline candidate embedding generation thereby minimizing the need for real-time transformer model execution. Consequently, our frame- work is able to scale to large e-commerce QA traffic. Extensive evaluation on user queries shows that our framework significantly outper- forms both syntactic and semantic baselines in offline as well as large scale online A/B setups of a popular e-commerce service. 1 Introduction Product pages on an e-commerce service (eg. Ama- zon) are often overloaded with information. Cus- tomers wanting to search for a piece of specific information about a product find it difficult to sift ∗This work was done while author was in Community Shopping team. through. To address this issue most services pro- vide an instant QA system on the product pages enabling users to type their query and get instant an- swers curated from various sources present on the page. Figure 1 shows the QA widget on Amazon, and the three sources viz. Product information (eg: bullet points, technical specifications etc.), Cus- tomer Q&A’s (where customers/sellers provide an answer to the posted questions by customers, hence- forth called community QA or CQA section), and Customer reviews from where a response is gener- ated. In this paper, we focus on retrieving responses Figure 1: Instant QA widget on Amazon from the CQA section. Hence our goal is to learn a robust relevance function between user queries and CQA pairs. Notably, these two domains differ sig- nificantly in language characteristics. User queries are typically short, often ill-formed and incomplete, whereas CQA pairs tend to be more complete and well-formed. For example, "Bettry perfon" is a user 4009 query where the intended question probably was "how is the battery performance?". Furthermore, we analyzed CQA section along with 3 months user query logs of …

Content truncated to 500 words