Learning Emphasis Selection for Written Text in Visual Media from Crowd-Sourced Label Distributions

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1167–1172 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 1167 Learning Emphasis Selection for Written Text in Visual Media from Crowd-Sourced Label Distributions Amirreza Shirani†, Franck Dernoncourt‡, Paul Asente‡, Nedim Lipka‡, Seokhwan Kim§, Jose Echevarria‡ and Thamar Solorio† †University of Houston ‡Adobe Research †{ashirani,tsolorio}@uh.edu§Amazon Alexa AI ‡{dernonco,asente,lipka,echevarr}@adobe.com §seokhwk@amazon.com Abstract In visual communication, text emphasis is used to increase the comprehension of writ- ten text and to convey the author’s intent. We study the problem of emphasis selec- tion, i.e. choosing candidates for emphasis in short written text, to enable automated de- sign assistance in authoring. Without know- ing the author’s intent and only considering the input text, multiple emphasis selections are valid. We propose a model that employs end-to-end label distribution learning (LDL) on crowd-sourced data and predicts a selection distribution, capturing the inter-subjectivity (common-sense) in the audience as well as the ambiguity of the input. We compare the model with several baselines in which the problem is transformed to single-label learning by map- ping label distributions to absolute labels via majority voting. 1 Introduction Visual communication relies heavily on images and short texts. Whether it is flyers, posters, ads, social media posts or motivational messages, it is usually highly designed to grab a viewer’s atten- tion and convey a message in the most efficient way. For text, word emphasis is used to capture the intent better, removing the ambiguity that may exist in some plain texts. Word emphasis can clar- ify or even change the meaning of a sentence by drawing attention to some specific information. It can be done with colors, backgrounds, or fonts, or with styles like italic and boldface. Some graphic design applications such as Adobe Spark1 perform automatic text layout using templates that include images and text with differ- ent fonts and colors. However, their text layout algorithms are mainly driven by visual attributes like word length, rather than the semantics of the 1https://spark.adobe.com (a) (b) Figure 1: Two different text layouts emphasizing dif- ferent parts of the sentence. text or the user’s intent, which can lead to un- intended emphasis and the wrong message. Fig- ure 1a shows an example that is aesthetically ap- pealing but fails to effectively communicate its in- tent. Understanding the text would allow the sys- tem to propose a different layout that emphasizes words that contribute more to the communication of the intent, as shown in Figure 1b. We investigate models that aim to understand the most common interpretation of a short piece of text, so the right emphasis can be achieved au- tomatically or interactively. The ultimate goal is to enable design assistance for the user during au- thoring. The main focus is on short text instances for social media, with a variety of examples from inspirational quotes to advertising slogans. We model emphasis using plain text with no additional context from the user or the rest of the design. …

Content truncated to 500 words