Corpus Generation for Voice Command in Smart Home and the Effect of Speech Synthesis on End-to-End SLU

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 6395–6404 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 6395 Corpus generation for voice command in smart home and the effect of speech synthesis on End-to-End SLU Thierry Desot, Franc¸ois Portet, Michel Vacher Univ. Grenoble Alpes, CNRS, Grenoble INP, LIG 38000 Grenoble, France Thierry.Desot@univ-grenoble-alpes.fr, {Francois.Portet,Michel.Vacher}@imag.fr Abstract Massive amounts of annotated data greatly contributed to the advance of the machine learning field. However such large data sets are often unavailable for novel tasks performed in realistic environments such as smart homes. In this domain, semantically annotated large voice command corpora for Spoken Language Understanding (SLU) are scarce, especially for non-English languages. We present the automatic generation process of a synthetic semantically-annotated corpus of French commands for smart-home to train pipeline and End-to-End (E2E) SLU models. SLU is typically performed through Automatic Speech Recognition (ASR) and Natural Language Understanding (NLU) in a pipeline. Since errors at the ASR stage reduce the NLU performance, an alternative approach is End-to-End (E2E) SLU to jointly perform ASR and NLU. To that end, the artificial corpus was fed to a text-to-speech (TTS) system to generate synthetic speech data. All models were evaluated on voice commands acquired in a real smart home. We show that artificial data can be combined with real data within the same training set or used as a stand-alone training corpus. The synthetic speech quality was assessed by comparing it to real data using dynamic time warping (DTW). Keywords: Spoken language understanding, automatic speech recognition, natural language understanding, corpora and language resources, ambient intelligence, voice-user interface, text-to-speech, dynamic time warping 1. Introduction Smart homes with integrated voice-user interfaces (VUI) can provide in-home assistance to older adults (Peetoom et al., 2015), allowing them to retain autonomy and pro- viding swift intervention in emergency situations through distant interaction (Vacher et al., 2015). However these systems include multiple modules, such as a Spoken Lan- guage Understanding (SLU) module that must be able to extract the intent of the user from the voice command and its named entities. The intent reflects the intention of the speaker whereas entities and relations are called slots and represent the pieces of information that are relevant for the given task (Tur and De Mori, 2011). SLU systems typically consist of a pipeline of automatic speech recognition (ASR) and natural language understanding (NLU) modules. The NLU module takes as input a transcript of the voice com- mand provided by the ASR module and extracts its mean- ing in a form that can be processed by a Decision Making module. The NLU model is trained on clean transcriptions whereas erroneous ASR transcriptions reduce the SLU per- formances. Different from pipeline SLU, the E2E SLU ap- proach combines ASR and NLU in one model and avoids cumulative ASR and NLU errors (Ghannay et al., 2018). In (Qian et al., 2017) it is demonstrated that E2E SLU does not necessarily require an initial step of ASR if …

Content truncated to 500 words