Paraphrase Acquisition from Image Captions

Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 3348–3358 May 2-6, 2023 ©2023 Association for Computational Linguistics Paraphrase Acquisition from Image Captions Marcel Gohsen Bauhaus-Universität Weimar Matthias Hagen Friedrich-Schiller-Universität Jena Martin Potthast Leipzig University and ScaDS.AI Benno Stein Bauhaus-Universität Weimar Abstract We propose to use image captions from the Web as a previously underutilized resource for paraphrases (i.e., texts with the same “mes- sage”) and to create and analyze a correspond- ing dataset. When an image is reused on the Web, an original caption is often assigned. We hypothesize that different captions for the same image naturally form a set of mutual paraphrases. To demonstrate the suitability of this idea, we analyze captions in the English Wikipedia, where editors frequently relabel the same image for different articles. The paper in- troduces the underlying mining technology, the resulting Wikipedia-IPC dataset, and compares known paraphrase corpora with respect to their syntactic and semantic paraphrase similarity to our new resource. In this context, we introduce characteristic maps along the two similarity di- mensions to identify the style of paraphrases coming from different sources. An annotation study demonstrates the high reliability of the al- gorithmically determined characteristic maps. 1 Introduction Two texts that convey “semantically equivalent in- formation” via a different wording are called para- phrases, or said to be in a paraphrase relation. A variety of natural language processing tasks have been approached by using paraphrase resources or paraphrase generation and detection algorithms: textual entailment (Dagan et al., 2013; Marelli et al., 2014; Izadinia et al., 2015), semantic similarity (Agirre et al., 2015; Li and Srikumar, 2016), ma- chine translation (Mehdizadeh Seraj et al., 2015), text reformatting (Stein et al., 2014), or question an- swering (Fader et al., 2013). Paraphrasing can help to improve sentence compression and text summa- rization (Cordeiro et al., 2007) as well as natural language understanding (Ganitkevitch, 2013). To tackle the task of paraphrasing, large amounts of paraphrases usually are helpful. Hence, para- phrase acquisition has gained popularity over the Flourine-containing durable water repellent makes a fabric water- resistant. Water bead on a fabric that has been made non-wetting by chemical treatment. Figure 1: Paraphrased captions of an image in the Wikipedia pages ‘Durable water repellent’ (left) and ‘Wetting’ (right). recent years and paraphrase datasets have been cre- ated manually, (semi-)automatically, and via dis- tant supervision resulting in different levels of “nat- uralness”, diversity (topic, genre, language, etc.), granularity (word, sentence, or passage level), and scale (see Section 2 for details). For model training, Zhang et al. (2019b) strongly emphasize the impor- tance of negative examples that are lexically similar but semantically inappropriate as paraphrases. With image captions, we propose an as yet un- derutilized source of natural paraphrases. The un- derlying hypothesis is that different captions for the same image form paraphrase candidates be- cause the captions may describe the same image content. We study the phenomenon of ‘image reuse and captioning’ in the English Wikipedia (Figure 1 shows an …

Content truncated to 500 words