CogAlign: Learning to Align Textual Neural Representations to Cognitive Language Processing Signals

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 3758–3769 August 1–6, 2021. ©2021 Association for Computational Linguistics 3758 CogAlign: Learning to Align Textual Neural Representations to Cognitive Language Processing Signals Yuqi Ren and Deyi Xiong ∗ College of Intelligence and Computing, Tianjin University, Tianjin, China {ryq20, dyxiong}@tju.edu.cn Abstract Most previous studies integrate cognitive lan- guage processing signals (e.g., eye-tracking or EEG data) into neural models of natural language processing (NLP) just by directly concatenating word embeddings with cogni- tive features, ignoring the gap between the two modalities (i.e., textual vs. cognitive) and noise in cognitive features. In this pa- per, we propose a CogAlign approach to these issues, which learns to align textual neural representations to cognitive features. In Co- gAlign, we use a shared encoder equipped with a modality discriminator to alternatively encode textual and cognitive inputs to capture their differences and commonalities. Addition- ally, a text-aware attention mechanism is pro- posed to detect task-related information and to avoid using noise in cognitive features. Ex- perimental results on three NLP tasks, namely named entity recognition, sentiment analysis and relation extraction, show that CogAlign achieves significant improvements with mul- tiple cognitive features over state-of-the-art models on public datasets. Moreover, our model is able to transfer cognitive information to other datasets that do not have any cogni- tive processing signals. The source code for CogAlign is available at https://github. com/tjunlp-lab/CogAlign.git. 1 Introduction Cognitive neuroscience, from a perspective of lan- guage processing, studies the biological and cogni- tive processes and aspects that underlie the mental language processing procedures in human brains while natural language processing (NLP) teaches machines to read, analyze, translate and generate human language sequences (Muttenthaler et al., 2020). The commonality of language process- ing shared by these two areas forms the base of ∗Corresponding author cognitively-inspired NLP, which uses cognitive language processing signals generated by human brains to enhance or probe neural models in solving a variety of NLP tasks, such as sentiment analysis (Mishra et al., 2017; Barrett et al., 2018), named entity recognition (NER) (Hollenstein and Zhang, 2019), dependency parsing (Strzyz et al., 2019), relation extraction (Hollenstein et al., 2019a), etc. In spite of the success of cognitively-inspired NLP in some tasks, there are some issues in the use of cognitive features in NLP. First, for the inte- gration of cognitive processing signals into neural models of NLP tasks, most previous studies have just directly concatenated word embeddings with cognitive features from eye-tracking or EEG, ignor- ing the huge differences between these two types of representations. Word embeddings are usually learned as static or contextualized representations of words in large-scale spoken or written texts gen- erated by humans. In contrast, cognitive language processing signals are collected by specific medi- cal equipments, which record the activity of human brains during the cognitive process of language processing. These cognitive processing signals are usually assumed to represent psycholinguistic in- formation (Mathias et al., 2020) or …

Content truncated to 500 words