Countering Language Drift via Visual Grounding

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 4385–4395, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 4385 Countering Language Drift via Visual Grounding Jason Lee†, Kyunghyun Cho†‡⋆, Douwe Kiela‡ † New York University; ⋆ CIFAR Azrieli Global Scholar; ‡ Facebook AI Research jasonlee@cs.nyu.edu, kyunghyun.cho@nyu.edu, dkiela@fb.com Abstract Emergent multi-agent communication proto- cols are very different from natural language and not easily interpretable by humans. We find that agents that were initially pretrained to produce natural language can also experi- ence detrimental language drift: when a non- linguistic reward is used in a goal-based task, e.g. some scalar success metric, the commu- nication protocol may easily and radically di- verge from natural language. We recast trans- lation as a multi-agent communication game and examine auxiliary training constraints for their effectiveness in mitigating language drift. We show that a combination of syntactic (lan- guage model likelihood) and semantic (visual grounding) constraints gives the best com- munication performance, allowing pre-trained agents to retain English syntax while learning to accurately convey the intended meaning. 1 Introduction A long-standing goal of artificial intelligence re- search is to develop agents that can cooperate with other agents, including humans, to solve tasks. As Gauthier and Mordatch (2016) propose, one way to get closer to this goal is to develop agents that can flexibly use human language to coordinate with themselves and with humans. Recently, there has been a renewed interest in multi-agent communication (Foerster et al., 2016; Lazaridou et al., 2016). While agents can be very effective in solving the tasks that they were trained on, their multi-agent communication protocols bear little resemblance to human languages. A major open question revolves around training multi-agent systems such that their communication protocols can be interpreted by humans. One option is to pre-train in a supervised fashion with human language, but even then it is found that the protocols diverge quickly when the agents are Intended message: 2 elephants and 1 lion No constraints floopy globber Syntactic democracy is a politi- cal system Syntactic+Semantic a pair of elephants and a large feline Table 1: Examples of valid messages under different con- straints. Without constraints, agents may invent an arbitrary protocol to communicate the intended message. Syntactic constraints enforce “Englishness” but not semantic correspon- dence. Semantic constraints, e.g. with visual grounding, can enforce the communication channel to retain the meaning of the message. fine-tuned on an external reward, as Lewis et al. (2017) showed on a negotiation task. Indeed, lan- guage drift is to be expected if we are optimizing for an external non-linguistic reward, such as a reward based on whether or not two agents success- fully accomplish a negotiation. Language drift might be avoided by imposing a “naturalness” constraint, e.g. by factoring language model likelihood into the reward function. How- ever, such a constraint only acts on the syntax of the generated language, ignoring its semantics. See Table 1 for an …

Content truncated to 500 words