Investigating Cross-lingual Multi-level Adaptive Networks: The Importance of the Correlation of Source and Target Languages

Investigating Cross-lingual Multi-level Adaptive Networks: The Importance of the Correlation of Source and Target Languages Alexandros Lazaridisa,*, Ivan Himawanb, Petr Motliceka, Iosif Mporasc and Philip N. Garnera aIdiap Research Institute, Martigny, Switzerland bScience and Engineering Faculty, Queensland University of Technology, Australia cSchool of Engineering and Technology, University of Hertfordshire, Hertfordshire, UK alaza@idiap.ch Abstract The multi-level adaptive networks (MLAN) technique is a cross-lingual adaptation framework where a bottleneck (BN) layer in a deep neural network (DNN) trained in a source lan- guage is used for producing BN features to be exploited in a second DNN in a target language. We investigate how the correlation (in the sense of phonetic similarity) of the source and target languages and the amount of data of the source language affect the efficiency of the MLAN schemes. We experiment with three different scenarios using, i) French, as a source language uncorrelated to the target language, ii) Ukrainian, as a source language correlated to the target one and finally iii) English as a source language uncorrelated to the target language using a relatively large amount of data in respect to the other two scenarios. In all cases Russian is used as target language. GLOBALPHONE data is used, except for English, where a mixture of LIBRISPEECH, TEDLIUM and AMIDA is available. The results have shown that both of these two factors are important for the MLAN schemes. Specifically, on the one hand, when a modest amount of data from the source language is used, the correlation of the source and target languages is very important. On the other hand, the correlation of the two languages seems to be less important when a relatively large amount of data, from the source language, is used. The best performance in word error rate (WER), was achieved when the English language was used as the source one in the multi-task MLAN scheme, achieving a relative improvement of 9.4% in respect to the baseline DNN model. 1. Introduction We are interested in general in multilingual automatic speech recognition (ASR), and in particular in its use when com- bined with machine translation for broadcast data monitor- ing. As news becomes pertinent in new parts of the world, different languages and dialects become relevant. It is neces- sary for already multilingual ASR systems to adapt to these new environments as quickly as possible. When news is happening in an under-resourced language (or dialect), data may be available, or may be collected, in a closely related (correlated) language. It seems reasonable to assume that such data would be useful in ASR for the under- resourced language. One successful class of approaches for cross-lingual adaptation has made use of posterior features derived from neural networks in tandem and hybrid ASR systems [1, 2, 3]. In these systems, features derived from a neural network trained as a phone classifier (i.e., bottleneck fea- tures) are concatenated with the traditional spectral features (e.g., MFCCs, PLPs) in order to train ASR systems. It has been shown in many studies that the …

Content truncated to 500 words