Nkululeko: A Tool For Rapid Speaker Characteristics Detection

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 1925–1932 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 1925 Nkululeko: A Tool For Rapid Speaker Characteristics Detection Felix Burkhardt1,2, Johannes Wagner1, Hagen Wierstorf1, Florian Eyben1, Bj¨orn Schuller1,3,4 1audEERING GmbH, Germany, 2Technical University of Berlin, Germany, 3Chair EIHW, Universit¨at Augsburg, Germany, 4GLAM, Imperial College London, UK {fburkhardt, jwagner, hwierstorf, fe, bs}@audeering.com Abstract We present advancements with a software tool called Nkululeko, that lets users perform (semi-) supervised machine learning experiments in the speaker characteristics domain. It is based on audformat, a format for speech database metadata description. Due to an interface based on configurable templates, it supports best practise and very fast setup of experiments without the need to be proficient in the underlying language: Python. The paper explains the handling of Nkululeko and presents two typical experiments: comparing the expert acoustic features with artificial neural net embeddings for emotion classification and speaker age regression. Keywords: tools, machine learning, speaker characteristics 1. Introduction and Related Work In the past decades, the research community has been confronted with the tremendous success of ap- proaches to estimate knowledge with artificial neural nets (ANN), predominantly under the label deep learn- ing (DL). Especially empirical sciences benefit from the opportunity to test hypotheses with machine learn- ing experiments that are able to analyse statistically very large data quantities. Many empirical researchers, phoneticians, and lin- guists, did not study computer science and struggle with the necessary programming skills to set machine learning experiments up. Nkululeko1 is being developed preliminary as a tool for a series of machine learning seminars at the institute for speech communication at the Technical University of Berlin to enable students to conduct machine learning experiments with a very flat learning curve by simply filling configuration files. This makes it much easier to be used compared to other high level frameworks for deep learning like Keras, Torch, Google AutoML, or end2you (Chollet, 2017; Chaudhary et al., 2020; Tzirakis et al., 2018; Bisong, 2019) while still keeping the flexibility as it is based on Torch and Keras. There are other frameworks that let users do data pro- cessing without the need to program too much code. Knime2 is probably the most prominent one: data pro- cessing chains can be specified by connecting prede- fined modules, many of them machine learning mod- els. In contrast to Nkululeko, Knime is a rather general data exploration software and not focused on audio pro- cessing and thus misses the acoustic feature extraction components as well as making it harder to set up spe- 1On the lookout for a distinctive name for this project we stumbled across an 1980ies punk album title. They tried new things out quickly, so this seemed fitting. 2https://www.knime.com/ cific experiments. In (Evain et al., 2021), a framework named Lebench- mark is described, that enables users to perform self-supervised learning (SSL) experiments with pre- trained models for French emotional speech. Although also meant to be used …

Content truncated to 500 words