Robust Target Speaker Tracking in Broadcast TV Streams

Computational Linguistics and Chinese Language Processing Vol. 11, No. 1, March 2006, pp. 57-72 57 © The Association for Computational Linguistics and Chinese Language Processing [Received June 15, 2005; Revised January 23, 2006; Accepted January 27, 2006] Robust Target Speaker Tracking in Broadcast TV Streams Junmei Bai*, Hongchen Jiang*, Shilei Zhang*, Shuwu Zhang* and Bo Xu* Abstract This paper addresses the problem of audio change detection and speaker tracking in broadcast TV streams. A two-pass audio change detection algorithm, which includes detection of the potential change boundaries and refinement, is proposed. Speaker tracking is performed based on the results of speaker change detection. In speaker tracking, Wiener filtering, endpoint detection of pitch, and segmental cepstral feature normalization are applied to obtain a more reliable result. The algorithm has low complexity. Our experiments show that the algorithm achieves very satisfactory results. Keywords: Speaker Tracking, Audio Segmentation, Entropy, GMM 1. Introduction Broadcast TV programs are rich multimedia information resources. They contain large amounts of AV (audio & video) contents including speech, music, images, motion, text, and so on. Finding ways to extract and manage these various kinds of AV content information is becoming extremely important and necessary for application-oriented multimedia content mining and management. The analysis and classification of audio data are important tasks in many applications, such as speaker tracking, speech recognition, and content-based indexing. Among of them, target speaker tracking in TV streams is an important research topic for TV scene analysis. In contrast with general speaker recognition, speaker detection in audio streams usually requires segments of relatively homogenous speech and speaker tracking in this task should also determine the target speakers’ locations, in other word, the starting and ending times. In such applications, effective methods for segmenting continuous audio streams * The High-Tech. Innovation Center, Institute of Automation, Chinese Academy of Sciences, Beijing, China. E-mail: {jmbai, hcjiang, slzhang, swzhang, xubo}@hitic.ia.ac.cn 58 Junmei Bai et al. into homogeneous segments are required. The problem of acoustic segmentation and classification has become crucial for the application of automatic speech recognition to audio stream processing. The automatic segmentation of long audio streams and the clustering of audio segments according to different acoustic characteristics have received much attention recently [Lu and Zhang 2002; Chen and Gopalakrishnan 1998; Delacourt and Wellekens 2000; Wilcox et al. 1994; Pietquin et al. 2002]. To detect target speakers in an audio stream, it is best to segment the audio stream into homogeneous regions according to changes in speaker identity, environmental conditions and channel conditions. In fact, there are no explicit cues of changes among these audio signals, and the same speaker may appear multiple times in audio streams. Thus, it is not easy to segment an audio stream correctly. Various segmentation algorithms proposed in the literature [Lu and Zhang 2002; Chen and Gopalakrishnan 1998; Delacourt and Wellekens 2000; Ajmera et al. 2003; Cettolo and Federico 2000] can be categorized as follows [Chen et al. 1998]: 1) Decoder-guided segmentation algorithms: The input audio stream is first decoded by an automation …

Content truncated to 500 words