Attention Understands Semantic Relations

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 4040–4050 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 4040 Attention Understands Semantic Relations Anastasia Chizhikova♠♡ Sanzhar Murzakhmetov♠ Oleg Serikov‡♠♡ Tatiana Shavrina‡$ Mikhail Burtsev♠‡ ♡ HSE University ♠ DeepPavlov lab, MIPT ‡ AIRI $ SberDevices Moscow, Russia apchizhikova@edu.hse.ru, murzakhmetov.s@phystech.edu, oserikov@hse.ru shavrina@airi.net, burtcev.ms@mipt.ru Abstract Today, natural language processing heavily relies on pre-trained large language models. Even though such models are criticised for poor interpretability, they still yield state-of-the-art solutions for a wide range of very different tasks. While many probing studies have been conducted to measure the models awareness of grammatical knowledge, semantic probing is less popular. In this work, we introduce a probing pipeline to study how semantic relations are represented in transformer language models. We show that in this task, attention scores express the information about relations similar to the layers’ output activations despite their lesser ability to represent surface cues. This supports the hypothesis that attention mechanisms focus not only on syntactic relational information but semantic as well. Keywords: ontology extraction, knowledge probing, semantic probing, explainable AI (XAI), language models inter- pretation, bertology 1. Introduction Present-day monopoly of foundation language models in the majority of NLP tasks forces researchers and practitioners to rely on popular large models without genuinely understanding the models’ behaviour. The development of such models, regardless of impressive results on the existing benchmarks (Wang et al., 2019; Shavrina et al., 2020), leads to an interpretability crisis too. For example, general pre-trained models are often criticised for memorising things instead of generalising over the training knowledge. To enlighten the mechanisms driving these black-box models’ behaviour, a research area of models interpre- tation has developed. Probing studies were conducted outlining the limits to which the learned behaviour of the existing language models agrees with those devel- oped by research linguists (Conneau and Kiela, 2018). The majority of probing works address grammar, while semantic and factual knowledge remains largely under- studied (Rogers et al., 2020). Given the above, there is a gap in understanding of how models capture factual and semantic knowledge. By interpreting the models which link texts to the ex- isting ontologies and databases, one could narrow this gap: models are expected to use factual and semantic knowledge to solve downstream tasks efficiently. The interpretation of such a model, namely a Relation Ex- traction model, is the key narrative of the present paper. In this work, we introduce an approach to studying lan- guage models’ factual and semantic knowledge repre- sentations. We do so by constructing an intentionally naive pipeline of Relation Extraction and then inter- preting its steps. We choose the BERT (Devlin et al., 2018) model as an object of our study, as it is the model which is the most covered with interpretability works and a reasonable baseline for the majority of current downstream tasks. We employ a probing methodol- ogy to study if the BERT’s intermediate inference steps provide enough information to identify the …

Content truncated to 500 words