ECOL-R: Encouraging Copying in Novel Object Captioning with Reinforcement Learning

Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics, pages 1222–1234 April 19 - 23, 2021. ©2021 Association for Computational Linguistics 1222 ECOL-R: Encouraging Copying in Novel Object Captioning with Reinforcement Learning Yufei Wang1 and Ian D. Wood1 and Stephen Wan2 and Mark Johnson3 Macquarie University, Sydney, Australia1 CSIRO Data61, Sydney, Australia2 Oracle Digital Assistant, Oracle Corporation3 yufei.wang@students.mq.edu.au, ian.wood@mq.edu.au stephen.wan@data61.csiro.au mark.mj.johnson@oracle.com Abstract Novel Object Captioning is a zero-shot Im- age Captioning task requiring describing ob- jects not seen in the training captions, but for which information is available from ex- ternal object detectors. The key challenge is to select and describe all salient detected novel objects in the input images. In this pa- per, we focus on this challenge and propose the ECOL-R model (Encouraging Copying of Object Labels with Reinforced Learning), a copy-augmented transformer model that is en- couraged to accurately describe the novel ob- ject labels. This is achieved via a specialised reward function in the SCST reinforcement learning framework (Rennie et al., 2017) that encourages novel object mentions while main- taining the caption quality. We further restrict the SCST training to the images where de- tected objects are mentioned in reference cap- tions to train the ECOL-R model. We addi- tionally improve our copy mechanism via Ab- stract Labels, which transfer knowledge from known to novel object types, and a Morpholog- ical Selector, which determines the appropri- ate inflected forms of novel object labels. The resulting model sets new state-of-the-art on the nocaps (Agrawal et al., 2019) and held-out COCO (Hendricks et al., 2016) benchmarks. 1 Introduction Novel Object Captioning is a zero-shot Image Cap- tioning task where the captions should mention novel objects (i.e., not seen in the training cap- tions), but for which information is available from external object detectors. To produce high-quality captions, the captioning models should select and describe all salient detected objects and avoid men- tioning minor or irrelevant details in the input im- ages. As shown in Figure 1, caption A is the best caption among the three because A mentions all salient objects in the images without any unneces- sary details while B mentions Bread which is just a Figure 1: Caption A is the ground-truth caption for the image. Compared with B and C, A is the best cap- tion because it mentions all salient objects (i.e, Ham- burger, French Fries and Drinks). We use Abstract La- bels, that is hypernyms of the objects’ detected object labels in the object representations, transferring knowl- edge from the objects seen in the training captions to novel objects. Our copy mechanism also selects appro- priate inflected forms of object labels (i.e., Hamburgers vs. Hamburger). minor detail; and C misses the salient object Drink. This paper aims to develop a captioning model that produces caption A. We use an advanced copy mechanism, similar to the one in Hu et al. (2020), to effectively integrate novel objects. We follow the setup in Agrawal et al. (2019) and use …

Content truncated to 500 words