Aspect Sentiment Classification with Aspect-Specific Opinion Spans

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 386–395, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 386 Dialogue Response Ranking Training with Large-Scale Human Feedback Data Xiang Gao Yizhe Zhang Michel Galley Chris Brockett Bill Dolan Microsoft Research, Redmond, WA, USA {xiag,yizzhang,mgalley,chrisbkt,billdol}@microsoft.com Abstract Existing open-domain dialog models are gen- erally trained to minimize the perplexity of tar- get human responses. However, some human replies are more engaging than others, spawn- ing more followup interactions. Current con- versational models are increasingly capable of producing turns that are context-relevant, but in order to produce compelling agents, these models need to be able to predict and opti- mize for turns that are genuinely engaging. We leverage social media feedback data (number of replies and upvotes) to build a large-scale training dataset for feedback prediction. To alleviate possible distortion between the feed- back and engagingness, we convert the rank- ing problem to a comparison of response pairs which involve few confounding factors. We trained DIALOGRPT, a set of GPT-2 based models on 133M pairs of human feedback data and the resulting ranker outperformed sev- eral baselines. Particularly, our ranker outper- forms the conventional dialog perplexity base- line with a large margin on predicting Reddit feedback. We finally combine the feedback prediction models and a human-like scoring model to rank the machine-generated dialog responses. Crowd-sourced human evaluation shows that our ranking method correlates bet- ter with real human preferences than baseline models.1 1 Introduction Conversing freely in natural language is one of the greatest challenges of artificial intelligence. End- to-end open-domain dialog systems have become increasingly powerful, with advanced model ar- chitectures and large-scale training (Zhang et al., 2019b; Adiwardana et al., 2020; Roller et al., 2020; Li et al., 2020). In some settings, human annotators 1Dataset and models open-sourced on https:// github.com/golsun/DialogRPT Figure 1: For many online communities, posts and comments have a tree structure and user can upvote or downvote each node individually. This allows us to de- fine measures (e.g. Width, Depth, and Updown) of hu- man feedback and build a large-scale training dataset for response quality prediction. cannot reliably distinguish between human- and machine-generated responses. Though surprisingly effective, the training objective for these models is conceptually simple: minimizing the perplexity of a reference response for a given context. However, a meaningful evaluation of response generation must take into account more than whether a generated turn is relevant in context, or whether it “sounds human.” Conventional neural conversation models often generate trivial or bland responses (Li et al., 2016; Zhao et al., 2017) that are relevant to context but are not engaging. Even human responses can vary dramatically in terms of tonal appropriateness and whether they are in- teresting enough to prompt a rich listener reaction. A successful dialog turn must be proactive, engag- ing, and consistent with social norms (Grice, 1975, 1989). In this work, we move beyond simple predic- tion of response relevance, augmenting this with a prediction of how likely a response is …

Content truncated to 500 words