CLeLfPC: a Large Open Multi-Speaker Corpus of French Cued Speech

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 987–994 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 987 CLeLfPC: a Large Open Multi-Speaker Corpus of French Cued Speech Brigitte Bigi(1), Maryvonne Zimmermann(2,3), Carine Andr´e(1) (1) LPL, CNRS, Aix-Marseille Univ, (2) ALPC, (3) Datha (1)13100 Aix-en-Provence, (2)75015 Paris, (3)94400 Vitry-sur-Seine brigitte.bigi@cnrs.fr, maryvonne.zimmermann@datha.io, carine.andre@univ-amu.fr Abstract Cued Speech is a communication system developed for deaf people to complement speechreading at the phonetic level with hands. This visual communication mode uses handshapes in different placements near the face in combination with the mouth movements of speech to make the phonemes of spoken language look different from each other. This paper describes CLeLfPC - Corpus de Lecture en Langue franc¸aise Parl´ee Compl´et´ee, a corpus of French Cued Speech. It consists in about 4 hours of audio and HD video recordings of 23 participants. The recordings are 160 different isolated ‘CV’ syllables repeated 5 times, 320 words or phrases repeated 2-3 times and about 350 sentences repeated 2-3 times. The corpus is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. It can be used for any further research or teaching purpose. The corpus includes orthographic transliteration and other phonetic annotations on 5 of the recorded topics, i.e. syllables, words, isolated sentences and a text. The early results are encouraging: it seems that 1/ the hand position has a high influence on the key audio duration; and 2/ the hand shape has not. Keywords: Cued Speech, corpus, multimodality 1. Introduction The production of speech naturally involves lip move- ments; both the acoustic information as well as the lipreading are part of the phonological representation of hearing people. The processing of the audible acous- tic information is then influenced by the visual infor- mation (McGurk and MacDonald, 1976). For a bet- ter comprehension every sound of the language should look different but many sounds look alike on the lips when speaking. As a consequence, a large number of words have the same lip movements and can’t be distin- guished. The accuracy for lipreading sentences rarely exceeds 10%-30% words correct (R¨onnberg, 1995; R¨onnberg et al., 1998). In 1966, R. Orin Cornett invented the Cued Speech (Cornett, 1967), a visual system of communication; it adds information about the pronounced sounds that are not visible on the lips. Cued speech is a code to repre- sent each sound of a given language with a shape of the hand for a consonant, and a position around the face for a vowel. Actually, from both the hand position on the face and hand shapes, CV syllables are represented. So, a single CV syllable will be generated or decoded through both the lips position and the key of the hand. Each time a speaker pronounces a ’CV’ or ’V’ syllable, a cue is produced and other syllabic structures are pro- duced with several cues - for example, a ’CCV’ syllable is coded with the two keys ’C’ then ’CV’. When sounds look alike …

Content truncated to 500 words