Chinese Synesthesia Detection: New Dataset and Models

Findings of the Association for Computational Linguistics: ACL 2022, pages 3877 - 3887 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Chinese Synesthesia Detection: New Dataset and Models Xiaotong Jiang†, Qingqing Zhao‡, Yunfei Long♢, and Zhongqing Wang† ∗ † Natural Language Processing Lab, Soochow University, Suzhou, China ‡ Institute of Linguistics, Chinese Academy of Social Sciences, Beijing, China ♢ University of Essex, UK devjiang@outlook.com, zhaoqq@cass.org.cn yl20051@essex.ac.uk, wangzq@suda.edu.cn Abstract In this paper, we introduce a new task called synesthesia detection, which aims to extract the sensory word of a sentence, and to predict the original and synesthetic sensory modalities of the corresponding sensory word. Synesthe- sia refers to the description of perceptions in one sensory modality through concepts from other modalities. It involves not only a lin- guistic phenomenon, but also a cognitive phe- nomenon structuring human thought and ac- tion, which makes it become a bridge between figurative linguistic phenomenon and abstract cognition, and thus be helpful to understand the deep semantics. To address this, we con- struct a large-scale human-annotated Chinese synesthesia dataset, which contains 7,217 an- notated sentences accompanied by 187 sensory words. Based on this dataset, we propose a family of strong and representative baseline models. Upon these baselines, we further pro- pose a radical-based neural network model to identify the boundary of the sensory word, and to jointly detect the original and synesthetic sensory modalities for the word. Through ex- tensive experiments, we observe that the im- portance of the proposed task and dataset can be verified by the statistics and progressive per- formances. In addition, our proposed model achieves state-of-the-art results on the synes- thesia dataset. 1 Introduction Synesthesia refers to the association of perceptions in both perceptual experiences and language us- ages (Winter, 2019; Zhao, 2020). Synesthesia in perceptions, namely neurological synesthesia, de- scribes a special perceptual condition for specific people who can perceive colors from black-printed letters, touch sounds, taste shapes, and so forth (Cy- towic, 2002; Banissy et al., 2015). Synesthesia in language usages, named linguistic synesthesia al- ternatively, involves lexical items in one sensory ∗ Corresponding author 她用甜蜜蜜的语调说：那是我教他的喔！ (She said in a sweet tone, “That’s what I taught him!”) Taste Hearing Figure 1: An example of synesthetic sentence. modality to describe perceptions in another sen- sory modality (Ullmann, 1957; Williams, 1976). For instance, as shown in Figure 1, the gustatory adjective “甜蜜蜜” (sweet) can be used to describe an auditory perception, as in the phrase “甜蜜蜜的 语调” (a sweet tone). Different from extensive studies on synesthesia in neurological and linguistic areas, synesthesia has received little attention in natural language processing (NLP). One of the related topics in NLP is metaphor detection, which aims at identify- ing metaphorical expressions using computational models (Turney et al., 2011; Chen et al., 2020; Su et al., 2020). That is, synesthesia involves not only a linguistic phenomenon, but also a cognitive phe- nomenon structuring human thought. Naturally, synesthesia can bridge the gap between figurative linguistic phenomenon and abstract cognition on deep semantics. Thus, it may help …

Content truncated to 500 words