Causality-Guided Multi-Memory Interaction Network for Multivariate Stock Price Movement Prediction

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 1192–1215 July 9-14, 2023 ©2023 Association for Computational Linguistics A Theory of Unsupervised Speech Recognition Liming Wang1, Mark Hasegawa-Johnson1 and Chang D. Yoo2 1University of Illinois Urbana-Champaign 2Korea Advanced Institute of Science Technology {lwang114,jhasegaw}@illinois.edu, cd_yoo@kaist.ac.kr Abstract Unsupervised speech recognition (ASR-U) is the problem of learning automatic speech recognition (ASR) systems from unpaired speech-only and text-only corpora. While var- ious algorithms exist to solve this problem, a theoretical framework is missing to study their properties and address such issues as sensitivity to hyperparameters and training instability. In this paper, we proposed a general theoretical framework to study the properties of ASR-U systems based on random matrix theory and the theory of neural tangent kernels. Such a framework allows us to prove various learnabil- ity conditions and sample complexity bounds of ASR-U. Extensive ASR-U experiments on synthetic languages with three classes of transi- tion graphs provide strong empirical evidence for our theory (code available at cactuswith- thoughts/UnsupASRTheory.git). 1 Introduction Unsupervised speech recognition (ASR-U) is the problem of learning automatic speech recognition (ASR) systems from unpaired speech-only and text- only corpora. Such a system can not only signifi- cantly reduce the amount of annotation resources required for training state-of-the-art ASR system, but serve as a bridge between spoken and written language understanding tasks in the low-resource setting. Since its first proposal (Liu et al., 2018), it has seen remarkable progress and the current best system (Baevski et al., 2021) has achieved compara- ble performance to systems trained with paired data on various languages. However, there are several mysteries surrounding ASR-U, which potentially hinder the future development of such systems. In particular, prior experiments have found that training the current state-of-the-art ASR-U model, wav2vec-U (Baevski et al., 2021), requires care- ful tuning over the weights of various regulariza- tion losses to avoid converging to bad local op- tima and that even despite extensive regularization weight tuning, wav2vec-U may still fail to converge (Ni et al., 2022). Therefore, it remains a mystery whether or when unpaired speech and text data in- deed provide sufficient information for learning an ASR system. Another mystery is whether the suc- cess of existing ASR-U models based on generative adversarial net (GAN) (Goodfellow et al., 2014) is sufficiently explained by the GAN objective func- tion per se, or also requires other factors, such as randomness in training, quirks in the data used and careful domain-specific hyper-parameter settings, etc. In this paper, we provide a theoretical analysis of ASR-U to investigate the mysteries surrounding ASR-U. First, we prove learnability conditions and sample complexity bounds that crucially depend on the eigenvalue spacings of the transition prob- ability matrix of the spoken language. Random matrix theory shows that such learnability condi- tions are achievable with high probability. Next, we study the gradient flow of GAN-based ASR-U and provide conditions under which the generator min- imizing the GAN objective converges to the true generator. Finally, to verify …

Content truncated to 500 words