DP-Parse: Finding Word Boundaries from Raw Speech with an Instance Lexicon

DP-Parse: Finding Word Boundaries from Raw Speech with an Instance Lexicon Robin Algayres1,3,5 Tristan Ricoul3 Julien Karadayi3 Hugo Laurenc¸on3 Salah Zaiem3 Abdelrahman Mohamed2 Benoˆıt Sagot4 Emmanuel Dupoux1,3,4,5 Meta AI Research, France1, Meta AI Research, USA2, ENS/PSL, Paris, France3, EHESS, Paris, France4, Inria, Paris, France5, {robin.algayres, benoit.sagot}@inria.fr dpx@fb.com Abstract Finding word boundaries in continuous speech is challenging as there is little or no equivalent of a ‘space’ delimiter between words. Popu- lar Bayesian non-parametric models for text segmentation (Goldwater et al., 2006, 2009) use a Dirichlet process to jointly segment sen- tences and build a lexicon of word types. We introduce DP-Parse, which uses similar prin- ciples but only relies on an instance lexicon of word tokens, avoiding the clustering errors that arise with a lexicon of word types. On the Zero Resource Speech Benchmark 2017, our model sets a new speech segmentation state-of-the-art in 5 languages. The algorithm monotonically improves with better input representations, achieving yet higher scores when fed with weakly supervised inputs. Despite lacking a type lexicon, DP-Parse can be pipelined to a language model and learn semantic and syn- tactic representations as assessed by a new spoken word embedding benchmark. 1 1 Introduction One of the first tasks that infants face is to learn the words of their native language(s). To do so, they must solve a word segmentation problem, since words are rarely uttered in isolation but come up in multi-word utterances (Brent and Cartwright, 1996). Segmenting utterances into word or sub- word units is also an important step in NLP applications, where an input string of orthographic symbols is tokenized into words or sentence piece before being fed to a language model. While many writing systems use white spaces that make (ba- sic) tokenization a relatively simple task, others do not (e.g., Chinese) and turn tokenization into 1Our open-source implementation of DP-Parse: https:// gitlab.cognitive-ml.fr/ralgayres/dp-parse. a challenging machine learning problem (Li and Yuan, 1998). A similar situation arises for ‘text- less’ language models based on units derived from raw audio (Lakhotia et al., 2021), which, like in- fants, do not have access to isolated words or word-separating symbols. The most successful approach to unsupervised word segmentation for text inputs is based on non- parametric Bayesian models (Goldwater et al., 2006, 2009; Kawakami et al., 2019; Kamper et al., 2017b; Berg-Kirkpatrick et al., 2010; Eskander et al., 2016; Johnson et al., 2007; Godard et al., 2018) that jointly segment utterances and build a lexicon of frequent word forms using a Dirichlet process. The intuition is that frequent word forms function as anchors in a sentence, enabling the segmentation of novel words (like ‘dax’ in ‘did you see the dax in the street?’). Such models are tested on phonemized texts obtained by converting text into a stream of phonemes af- ter removing spaces and punctuation marks. Even though phonemized text may seem a reasonable approximation of continuous speech, such mod- els have given disappointing results when applied directly to speech inputs. Since the direct compar- ison between speech-based …

Content truncated to 500 words