What’s in Your Head? Emergent Behaviour in Multi-Task Transformer Models

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 8201–8215 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 8201 What’s in Your Head? Emergent Behaviour in Multi-Task Transformer Models Mor Geva1,2 Uri Katz1 Aviv Ben-Arie3 Jonathan Berant1,2 1Blavatnik School of Computer Science, Tel-Aviv University 2Allen Institute for Artificial Intelligence 3Independent Researcher {morgeva,urikatz1}@mail.tau.ac.il, avivba@gmail.com, joberant@cs.tau.ac.il Abstract The primary paradigm for multi-task training in natural language processing is to represent the input with a shared pre-trained language model, and add a small, thin network (head) per task. Given an input, a target head is the head that is selected for outputting the final prediction. In this work, we examine the be- haviour of non-target heads, that is, the output of heads when given input that belongs to a dif- ferent task than the one they were trained for. We find that non-target heads exhibit emergent behaviour, which may either explain the target task, or generalize beyond their original task. For example, in a numerical reasoning task, a span extraction head extracts from the in- put the arguments to a computation that results in a number generated by a target generative head. In addition, a summarization head that is trained with a target question answering head, outputs query-based summaries when given a question and a context from which the answer is to be extracted. This emergent behaviour suggests that multi-task training leads to non- trivial extrapolation of skills, which can be har- nessed for interpretability and generalization. 1 Introduction The typical framework for training a model in natu- ral language processing to perform multiple tasks is to have a shared pre-trained language model (LM), and add a small, compact neural network, often termed head, on top of the LM, for each task (Clark et al., 2019; Liu et al., 2019b; Nishida et al., 2019; Hu and Singh, 2021). The heads are trained in a supervised manner, each on labelled data collected for the task it performs (Devlin et al., 2019). At inference time, the output is read out of a selected target head, while the outputs from the other heads are discarded (Figure 1). What is the nature of predictions made by non- target heads given inputs directed to the target head? One extreme possibility is that the pre- pre-trained LM target head non-target head ? prediction shared representation input ... ... Figure 1: An illustration of multi-task training with a pre-trained LM. Given an input for one of the tasks, a shared representation is computed with a pre-trained LM (green). The target head outputs the prediction, while the other heads are ignored. In this work, we characterize the behaviour of the non-target head. trained LM identifies the underlying task, and con- structs unrelated representations for each task. In this case, the output of the non-target head might be arbitrary, as the non-target head observes inputs considerably different from those it was trained on. Conversely, the pre-trained LM might create simi- lar representations for all tasks, which can lead …

Content truncated to 500 words