Learning to Ask for Conversational Machine Learning

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 4164–4174, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 4164 Learning to Ask for Conversational Machine Learning Shashank Srivastava1∗ Igor Labutov2 Tom Mitchell3 1UNC Chapel Hill, 2LAER AI 3Machine Learning Department, Carnegie Mellon University ssrivastava@cs.unc.edu, igor.labutov@laer.ai, tom.mitchell@cmu.edu Abstract Natural language has recently been increas- ingly explored as a medium of supervision for training machine learning models. Here, we explore learning classification tasks using lan- guage in a conversational setting – where the automated learner does not simply receive lan- guage input from a teacher, but can proactively engage the teacher by asking template-based questions. We experiment with a reinforce- ment learning framework, where the learner’s actions correspond to question types and the reward for asking a question is based on how the teacher’s response changes performance of the resulting machine learning model on the learning task. In this framework, learning good question-asking strategies corresponds to asking sequences of questions that maximize the cumulative (discounted) reward, and hence quickly lead to effective classifiers. Empirical analysis shows that learned question-asking strategies can expedite classifier training by asking appropriate questions at different points in the learning process. The approach allows learning using a blend of strategies, including learning from observations, explanations and clarifications. 1 Introduction The ability to learn new tasks and behaviors from language is characteristic of human intelligence. In recent years, the fields of machine learning and NLP have seen an renewed interest in incorpo- rating natural language supervision in models of machine intelligence (Narasimhan et al., 2015; El- hoseiny et al., 2013; Goldwasser and Roth, 2014; Andreas et al., 2018; Fried et al., 2018; Wang et al., 2016). In particular, methods such as Bab- bleLabble (Hancock et al., 2018) and LNL (Srivas- tava et al., 2017) show progress towards realistic ∗* Work done while the first and second authors were at CMU Identify all important emails in my inbox Can you give me an explanation of what important emails might look like? Emails from nlp-corp.com are usually important Did you mean that emails where the sender’s address has a string match with “nlp-corp.com” are usually important? Yes, that’s right Can you label this new email as important/not important? This email is not important Asking for Explanations Requesting clarification Seeking labels Figure 1: Question-Answer dialog can enable learning from a mix of strategies, including label observations (traditional supervised learning), explanations and clar- ifications (to overcome parsing limitations). The out- put from the teacher-learner interaction is a classifica- tion model (here, for important emails). We present a framework that (a) enables learning classifiers from a mix of such supervision; (b) learns to ask appropriate sequences of questions to accelerate this. applications of supervised learning from language on tasks such as information extraction and email categorization. However, until now, such methods have been limited in two ways. First, despite a body of work on leveraging lan- guage …

Content truncated to 500 words