Filtering Speaker-Specific Words from Electronic Discussions

Filtering Speaker-Specific Words from Electronic Discussions Ingrid Zukerman and Yuval Marom School of Computer Science and Software Engineering Monash University, Clayton, Victoria 3800, AUSTRALIA � ingrid, yuvalm � @csse.monash.edu.au Abstract The work presented in this paper is the first step in a project which aims to cluster and sum- marise electronic discussions in the context of help-desk applications. The eventual objective of this project is to use these summaries to as- sist help-desk users and operators. In this pa- per, we identify features of electronic discus- sions that influence the clustering process, and offer a filtering mechanism that removes unde- sirable influences. We tested the clustering and filtering processes on electronic newsgroup dis- cussions, and evaluated their performance by means of two experiments: coarse-level cluster- ing and simple information retrieval. Our eval- uation shows that our filtering mechanism has a significant positive effect on both tasks. 1 Introduction The ability to draw on past experience is often use- ful in information-providing applications. For in- stance, users who interact with help-desk applica- tions would benefi t from the availability of rele- vant contextual information about their request, e.g., from previous, similar interactions between the sys- tem and other users, or from interactions between domain experts. The work reported in this paper is the fi rst step in a project which aims to provide such informa- tion. The eventual objective of our project is to au- tomatically identify related interactions in help-desk applications, and to generate summaries from their combined experience. These summaries would then assist both users and operators. Our approach to the identifi cation of related in- teractions hinges on the application of clustering techniques. These techniques have been used in Information Retrieval for some time (e.g. Salton, 1971). They involve grouping a set of related docu- ments, and then using a representative element to match input queries (as opposed to matching the whole collection of documents). Document cluster- ing has been used in search engine applications to improve and speed up retrieval (e.g. Zamir and Et- zioni, 1998), but also for more descriptive purposes, such as using representative elements of a cluster to generate lists of keywords (Neto et al., 2000). However, discussions (and dialogues in general) have distinguishing features which make cluster- ing a corpus of such interactions a more challeng- ing task than clustering plain documents. These features are: (1) the corpus consists of contribu- tions made by a community of authors, or “speak- ers”; (2) certain speakers are more dominant in the corpus; and (3) speakers often use idiosyncratic, speaker-specifi c language, or make comments that are not about the task at hand. In this paper, we report on a preliminary study where we cluster discussions carried out in elec- tronic newsgroups. Specifi cally, we report on the influence of the above features on the clustering pro- cess, and describe a fi ltering mechanism that identi- fi es and removes undesirable influences. Table 1 shows the newsgroups used as data sources in our experiments. These newsgroups …

Content truncated to 500 words