Automatic Utterance Segmentation in Instant Messaging Dialogue

Proceedings of the Australasian Language Technology Workshop 2005, pages 241–249, Sydney, Australia, December 2005. Automatic Utterance Segmentation in Instant Messaging Dialogue Edward Ivanovic Department of Computer Science and Software Engineering University of Melbourne edwardi@csse.unimelb.edu.au Abstract Instant Messaging (IM) chat sessions are real-time, text-based conversations which can be analyzed using dialogue-act mod- els. Dialogue acts represent the seman- tic information of an utterance, however, messages must be segmented into utter- ances before classification can take place. We describe and compare two statistical methods for automatic utterance segmen- tation and dialogue-act classification in task-based IM dialogue. It is shown that IM messages can be automatically seg- mented and classified to a very high ac- curacy using statistical machine learning. 1 Introduction Dialogue acts are a useful first level of analysis for describing discourse structure as they represent the illocutionary force of utterances such as assertions and declarations. Early work on speech act theory by Austin (1962) and Searle (1979) has been ex- tended in dialogue acts to model the conversational functions that utterances can perform. Table 1 shows an example dialogue with utterance segments and dialogue acts. As illustrated in Table 1, some messages contain multiple utterances and thus require segmentation before each utterance can be classified as a dialogue act. Once utterances are classified, the dialogue- acts may then be used for subsequent tasks such as machine translation (Tanaka and Yokoo, 1999), dia- logue game detection (Levin et al., 1999), and, in the case of spoken dialogue, speech recognition (Stol- cke et al., 2000). Speaker Message Sanders [Hello Customer]CONVENTIONAL-OPENING, [thank you for contacting MSN Shopping]THANKING. [This is Sanders and I look forward to assist- ing you today]STATEMENT Sanders [How are you doing today?]OPEN-QUESTION Customer [good]STATEMENT, [thanks]THANKING Sanders [How may I help you today?]OPEN-QUESTION Table 1: An example of the beginning of a dia- logue in our corpus showing utterance boundaries and dialogue-act tags in superscript. Instant Messaging (IM) consists of two or more people typing messages to each other in real time on a line-by-line basis. Although IM dialogue can take place with a group of people simultaneously writing to each other, for the purposes of this study we assume only two-party dialogue. As described in Ivanovic (2005), sequences of words are grouped into three levels: the first level is a Turn, consisting of at least one Message, which consists of at least one Utterance, defined as follows: Turn: A dialogue participant normally writes one or more message then waits for the other participant to respond, hence taking turns in writing messages. Message: A message is defined as a group of words that are sent from one dialogue participant to the other participant as a single unit. This is usually achieved by typing a message and pressing the En- ter key or a ‘Send’ button on the client program. A single turn can span multiple messages. Utterance: An utterance can be thought of as one complete semantic unit with respect to dialogue acts. This can be a complete sentence or as …

Content truncated to 500 words