Learning to Classify Email into “Speech Acts”

Learning to Classify Email into "Speech Acts" William W. Cohen1 Vitor R. Carvalho2 Tom M. Mitchell1,2 1Center for Automated Learning & Discovery Carnegie Mellon University Pittsburgh, PA 15213 2Language Technology Institute Carnegie Mellon University Pittsburgh, PA 15213 Abstract It is often useful to classify email accord- ing to the intent of the sender (e.g., "pro- pose a meeting", "deliver information"). We present experimental results in learn- ing to classify email in this fashion, where each class corresponds to a verb- noun pair taken from a predefined ontol- ogy describing typical “email speech acts”. We demonstrate that, although this categorization problem is quite dif- ferent from “topical” text classification, certain categories of messages can none- theless be detected with high precision (above 80%) and reasonable recall (above 50%) using existing text-classification learning methods. This result suggests that useful task-tracking tools could be constructed based on automatic classifi- cation into this taxonomy. 1 Introduction In this paper we discuss using machine learn- ing methods to classify email according to the intent of the sender. In particular, we classify emails according to an ontology of verbs (e.g., propose, commit, deliver) and nouns (e.g., infor- mation, meeting, task), which jointly describe the “email speech act” intended by the email sender. A method for accurate classification of email into such categories would have many potential benefits. For instance, it could be used to help an email user track the status of ongoing joint activi- ties. Delegation and coordination of joint tasks is a time-consuming and error-prone activity, and the cost of errors is high: it is not uncommon that commitments are forgotten, deadlines are missed, and opportunities are wasted because of a failure to properly track, delegate, and prioritize sub- tasks. The classification methods we consider methods which could be used to partially auto- mate this sort of activity tracking. A hypothetical example of an email assistant that works along these lines is shown in Figure 1. Bill, Do you have any sample scheduling-related email we could use as data? -Steve Assistant announces: “new email request, priority unknown.” Sure, I’ll put some together shortly. -Bill Assistant: “should I add this new commitment to your to- do list?” Fred, can you collect the msgs from the CSPACE corpora tagged w/ the “meeting” noun, ASAP? -Bill Assistant: notices outgoing request, may take action if no answer is received promptly. Yes, I can get to that in the next few days. Is next Monday ok? -Fred Assistant: notices incoming commitment. “Should I send Fred a reminder on Monday?” Figure 1 - Dialog with a hypothetical email assistant that automatically detects email speech acts. Dashed boxes indicate outgoing messages. (Messages have been edited for space and anonymity.) 2 Related Work Our research builds on earlier work defining il- locutionary points of speech acts (Searle, 1975), and relating such speech acts to email and work- flow tracking (Winograd, 1987, Flores & Lud- low, 1980, Weigant et al, 2003). Winograd suggested that research explicating the speech-act based “language-action perspective” on human …

Content truncated to 500 words