Adversarial Speech Generation and Natural Speech Recovery for Speech Content Protection

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 7291–7297 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 7291 Adversarial Speech Generation and Natural Speech Recovery for Speech Content Protection Sheng Li1, Jiyi Li2, Qianying Liu3, Zhuo Gong4 1National Institute of Information and Communications Technology (NICT), Kyoto, Japan 2University of Yamanashi, Kofu, Japan 3Kyoto University, Kyoto, Japan 4The University of Tokyo, Tokyo, Japan Email:sheng.li@nict.go.jp, jyli@yamanashi.ac.jp, ying@nlp.ist.i.kyoto-u.ac.jp, gongzhuo@gavo.t.u-tokyo.ac.jp Abstract With the advent of the General Data Protection Regulation (GDPR) and increasing privacy concerns, the sharing of speech data is faced with significant challenges. Protecting the sensitive content of speech is the same important as the voiceprint. This paper proposes an effective speech content protection method by constructing a frame-by-frame adversarial speech generation system. We revisited the adversarial examples generating method in the recent machine learning field and selected the phonetic state sequence of sensitive speech for the adversarial examples generation. We build an adversarial speech collection. Moreover, based on the speech collection, we proposed a neural network-based frame-by-frame mapping method to recover the speech content by converting from the adversarial speech to the human speech. Experiment shows our proposed method can encode and recover any sensitive audio, and our method is easy to be conducted with publicly available resources of speech recognition technology. Keywords: speech content protection, adversarial example, speech recognition system, deep neural network 1. Introduction With the advance of voice-based human-computer in- teraction and the development of intelligent devices, such as the Amazon Echo and Apple’s Homepod, speech data have become a new dimension of big data. The collection and sharing of real-world speech data enable innovative services and products, such as Ap- ple’s Siri and Google Assistant, and foster studies on intelligent algorithms. However, privacy and security concerns may hinder the collection and sharing of real- world speech data. Therefore, with the advent of the General Data Protection Regulation (GDPR) and in- creasing privacy concerns, the sharing of speech data is faced with significant challenges (Nautsch and et al., 2019). Currently, researchers focus on how to protect the speaker’s identifiable information, which is represented as voiceprint (as analogous to fingerprints), contained in the speech. An attacker may commit spoofing at- tacks (Wu and et al., 2015) to the voice authentica- tion systems or reputation attacks, such as fake Obama speech (Suwajanakorn and et al., 2017). Several meth- ods (Justin and et al., 2015; Qian and et al., 2018; Sri- vastava and et al., 2019; Fang and et al., 2019) for anonymizing speakers’ identities have been proposed to address these problems. However, protecting the sensitive content of speech is the same important as the voiceprint. The sen- sitive speech content may be found within particu- lar keywords or keyphrases such as named entities (places, dates, locations, organizations, etc.), financial and medical details, dirty words, or the entire speech. Determining what would be considered sensitive infor- mation ultimately depends on the use-cases. Speech content privacy refers to the ability to conceal …

Content truncated to 500 words