Joint Concept Learning and Semantic Parsing from Natural Language Explanations

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1527–1536 Copenhagen, Denmark, September 7–11, 2017. c⃝2017 Association for Computational Linguistics Joint Concept Learning and Semantic Parsing from Natural Language Explanations Shashank Srivastava Igor Labutov Machine Learning DepartmentTom Mitchell Carnegie Mellon University Pittsburgh, PA 15217 ssrivastava@cmu.edu ilabutov@cs.cmu.edu tom.mitchell@cmu.edu Abstract Natural language constitutes a predomi- nant medium for much of human learn- ing and pedagogy. We consider the prob- lem of concept learning from natural lan- guage explanations, and a small number of labeled examples of the concept. For example, in learning the concept of a phish- ing email, one might say ‘this is a phishing email because it asks for your bank account number’. Solving this problem involves both learning to interpret open-ended nat- ural language statements, as well as learn- ing the concept itself. We present a joint model for (1) language interpretation (se- mantic parsing) and (2) concept learning (classification) that does not require label- ing statements with logical forms. Instead, the model prefers discriminative interpre- tations of statements in context of observ- able features of the data as a weak signal for parsing. On a dataset of email-related concepts, this approach yields across-the- board improvements in classification per- formance, with a 30% relative improve- ment in F1 score over competitive classifi- cation methods in the low data regime. 1 Introduction The ability to automatically learn concepts1 from examples is a core cognitive ability, with applica- tions across diverse domains. Examples of such concepts include the concept of a ‘negative review’ in product reviews, the concept of ‘check’ over the domain of game states in chess, the concept of ‘fraud’ in credit history analysis, etc. Concept learn- ing is generally approached using classification 1where a concept is any Boolean function on some domain of instances. Figure 1: Examples of concepts explained using natural language statements. methods that can automatically leverage regulari- ties in large amounts of labeled training data. How- ever, there are two shortcomings of this paradigm. First, labeling large amounts of data is unnatural compared to how a person might teach another per- son (e.g., a human secretary) in a similar situation. For example, for identifying emails about postdoc positions, a university professor might say ‘These inquiries usually seek a postdoc opportunity and include a CV’, rather than label scores of examples of such emails. Second, acquiring large quanti- ties of labeled data may be infeasible because of a long tail of concepts that are highly domain or user specific. For our example of a busy profes- sor, it might be relevant to teach concepts such as ‘postdoc seeking emails’, ‘course related questions from students’, etc. to an email assistant in order to better manage her/his inbox. However, these concepts might be irrelevant to a general user. On the other hand, humans can efficiently learn about new concepts and phenomena through lan- guage. In fact, verbal and written language form the basis for much of human learning and pedagogy, as reflected in text-books, …

Content truncated to 500 words