Learning with Contrastive Examples for Data-to-Text Generation

Proceedings of the 28th International Conference on Computational Linguistics, pages 2352–2362 Barcelona, Spain (Online), December 8-13, 2020 2352 Learning with Contrastive Examples for Data-to-Text Generation Yui Uehara∗ † Tatsuya Ishigaki∗† Kasumi Aoki◦ Keiichi Goshima†⋄ Hiroshi Noji† Ichiro Kobayashi†◦ Hiroya Takamura†• Yusuke Miyao†‡ †National Institute of Advanced Industrial Science and Technology, Japan ◦Ochanomizu University •Tokyo Institute of Technology ⋄Waseda University ‡The University of Tokyo {yui.uehara, ishigaki.tatsuya, noji, takamura.hiroya}@aist.go.jp {g1120501, koba}@is.ocha.ac.jp keiichi.goshima@aoni.waseda.jp yusuke@is.s.u-tokyo.ac.jp Abstract Existing models for data-to-text tasks generate fluent but sometimes incorrect sentences e.g., “Nikkei gains” is generated when “Nikkei drops” is expected. We investigate models trained on contrastive examples, that is, incorrect sentences or terms, in addition to correct ones to reduce such errors. We first create rules to produce contrastive examples from correct ones by replacing frequent crucial terms such as “gain” or “drop”. We then use learning methods with several losses that exploit contrastive examples. Experiments on the market comment generation task show that 1) exploiting contrastive examples improves the capability to generate sentences with better lexical choices, without degrading the fluency, 2) the choice of the loss function is an important factor because the performances of different metrics depend on the types of loss functions, and 3) the use of the examples produced by some specific rules further improves performance. Human evaluation also supports the effectiveness of using contrastive examples. 1 Introduction We address the task of generating market comments from stock prices as illustrated in Fig. 1. This can be seen as a data-to-text generation task. Recently, neural data-to-text generation has been studied in a wide range of domains such as biography (Lebret et al., 2016; Liu et al., 2018), sports recap (Wiseman et al., 2017; Puduppully et al., 2019a; Puduppully et al., 2019b; Iso et al., 2019; Gong et al., 2019), and market comments (Murakami et al., 2017; Aoki et al., 2018; Aoki et al., 2019). These models generate fluent sentences, but we often observed problematic generated sentences in terms of correctness. As shown in Fig. 1, the word gain is possibly generated, although the word drop or rebound is expected. The terms that express the fluctuation of stock prices are crucial because such errors could reverse the meaning of the sentence in the worst case. Similar issues have been seen in other generation tasks, such as machine translation or summarization. The known solutions are, for example, the use of alignments between input and output (Sennrich, 2017; Arthur et al., 2016) or copy mechanisms (See et al., 2017). However, they cannot be directly applied to our task because ours treat sequences of numerical values as an input. In this paper, we consider how to alleviate such errors by using contrastive examples, which are iden- tical to the correct examples except for a single word: Nikkei gained vs. Nikkei dropped. Learning with such examples provides models direct signals on the words that are not to be generated in addition to those to be generated. We propose a learning framework to examine how to use such …

Content truncated to 500 words