Signal in Noise: Exploring Meaning Encoded in Random Character Sequences with Character-Aware Language Models

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 7120 - 7134 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Signal in Noise: Exploring Meaning Encoded in Random Character Sequences with Character-Aware Language Models Mark Bo Chu Columbia University mbc2165@columbia.edu Bhargav Srinivasa Desikan École Polytechnique Fédérale de Lausanne bhargav.srinivasadesikan@epfl.ch Ethan O. Nadler Carnegie Observatories University of Southern California enadler@carnegiescience.edu Ruggerio L. Sardo Sapienza University of Rome losardor@gmail.com Elise Darragh-Ford Stanford University KIPAC & Department of Physics edarragh@stanford.edu Douglas Guilbeault University of California, Berkeley Haas Business School douglas.guilbeault@berkeley.edu Abstract Natural language processing models learn word representations based on the distributional hy- pothesis, which asserts that word context (e.g., co-occurrence) correlates with meaning. We propose that n-grams composed of random character sequences, or garble, provide a novel context for studying word meaning both within and beyond extant language. In particular, ran- domly generated character n-grams lack mean- ing but contain primitive information based on the distribution of characters they contain. By studying the embeddings of a large corpus of garble, extant language, and pseudowords using CharacterBERT, we identify an axis in the model’s high-dimensional embedding space that separates these classes of n-grams. Further- more, we show that this axis relates to structure within extant language, including word part-of- speech, morphology, and concept concreteness. Thus, in contrast to studies that are mainly lim- ited to extant language, our work reveals that meaning and primitive information are intrinsi- cally linked. 1 Introduction What primitive information do character sequences contain? Modern natural language processing is driven by the distributional hypothesis (Firth, 1957), which asserts that the context of a linguistic expression defines its meaning (Emerson, 2020). Because existing words—which represent an ex- tremely small fraction of the space of possible char- acter sequences—appear in context together, the distributional paradigm at this level is limited in its ability to study the meaning of and informa- tion encoded by arbitrary character level n-grams (word forms). Furthermore, state-of-the-art compu- tational language models operating within the dis- tributional paradigm, such as BERT (Devlin et al., 2019), are mainly trained on extant words. Yet, a plethora of insights into language learning have emerged from inquiries into language beyond ex- tant words, such as the grammatical errors and inference patterns that children exhibit when distin- guishing extant words from non-linguistic auditory signals, including emotional expressions, auditory gestures, and other forms of paralinguistic speech (Yang, 2006; Carey, 2000). We therefore propose that character n-grams (i.e., sequences of alpha- betic characters) outside the space of extant lan- guage can provide new insights into the meaning of words and how they are represented by these models, beyond that captured by word and sub- word-based distributional semantics alone. We ex- plore this by studying the embeddings of randomly generated character n-grams (referred to as garble), which contain primitive communicative informa- tion but are devoid of meaning, using the Charac- terBERT model (El Boukkouri et al., 2020). Such randomly generated character n-grams are textual analogues of paralinguistic vocalizations—vocal …

Content truncated to 500 words