Activity detection for information access to oral communication

Activity detection for information access to oral communication Klaus Ries and Alex Waibel∗ {ries|ahw}@cs.cmu.edu Interactive Systems Labs, Carnegie Mellon University, Pittsburgh, PA, 15213, USA Interactive Systems Labs, Universit¨at Karlsruhe, Fakult¨at f¨ur Informatik, 76128 Karlsruhe, Germany http://www.is.cs.cmu.edu/ http://werner.ira.uka.de ABSTRACT Oral communication is ubiquitous and carries important in- formation yet it is also time consuming to document. Given the development of storage media and networks one could just record and store a conversation for documentation. The question is, however, how an interesting information piece would be found in a large database. Traditional informa- tion retrieval techniques use a histogram of keywords as the document representation but oral communication may offer additional indices such as the time and place of the rejoinder and the attendance. An alternative index could be the ac- tivity such as discussing, planning, informing, story-telling, etc. This paper addresses the problem of the automatic de- tection of those activities in meeting situation and everyday rejoinders. Several extensions of this basic idea are being discussed and/or evaluated: Similar to activities one can define subsets of larger database and detect those automati- cally which is shown on a large database of TV shows. Emo- tions and other indices such as the dominance distribution of speakers might be available on the surface and could be used directly. Despite the small size of the databases used some results about the effectiveness of these indices can be obtained. Keywords activity, dialogue processing, oral communication, speech, information access ∗We would like to thank our lab, especially Klaus Zechner, Alon Lavie and Lori Levin for their discussions and support. We would also like to thank our sponsors at DARPA. Any opinions, findings and conclusions expressed in this material are those of the authors and may not reflect the views of DARPA, or any other party. . 1. INTRODUCTION Information access to oral communication is becoming an interesting research area since recording, storing and trans- mitting large amounts of audio (and video) data is feasible today. While written information is often available elec- tronically (especially since it is typically entered on com- puters) oral communication is usually only documented by constructing a new document in written form such as a tran- script (court proceedings) or minutes (meetings). Oral com- munications are therefore a large untapped resource, espe- cially if no corresponding written documents are available and the cost of documentation using traditional techniques is considered high: Tutorial introductions by a senior staff member might be worthwhile to attend by many newcomers, office meetings may contain informations relevant for oth- ers and should be reproducable, informal and formal group meetings may be interesting but not fully documented. In essence the written form is already a reinterpretation of the original rejoinder. Such a reinterpretation are used to • extract and condense information • add or delete information • change the meaning • cite the rejoinder • relate rejoinders to each other Reinterpretation is a time consuming, expensive and op- tional step and written documentation is combining reinter- pretation and …

Content truncated to 500 words