A Deep Factorization of Style and Structure in Fonts

A Deep Factorization of Style and Structure in Fonts Nikita Srivatsan1 Jonathan T. Barron2 Dan Klein3 Taylor Berg-Kirkpatrick4 1Language Technologies Institute, Carnegie Mellon University, nsrivats@cmu.edu 2Google Research, barron@google.com 3Computer Science Division, University of California, Berkeley, klein@cs.berkeley.edu 4Computer Science and Engineering, University of California, San Diego, tberg@eng.ucsd.edu Figure 1: Example fonts from the Capitals64 dataset. The task of font reconstruction involves generating missing glyphs from partially observed novel fonts. Abstract We propose a deep factorization model for ty- pographic analysis that disentangles content from style. Specifically, a variational infer- ence procedure factors each training glyph into the combination of a character-specific con- tent embedding and a latent font-specific style variable. The underlying generative model combines these factors through an asymmet- ric transpose convolutional process to gener- ate the image of the glyph itself. When trained on corpora of fonts, our model learns a man- ifold over font styles that can be used to an- alyze or reconstruct new, unseen fonts. On the task of reconstructing missing glyphs from an unknown font given only a small num- ber of observations, our model outperforms both a strong nearest neighbors baseline and a state-of-the-art discriminative model from prior work. 1 Introduction One of the most visible attributes of digital lan- guage data is its typography. A font makes use of unique stylistic features in a visually consistent manner across a broad set of characters while pre- serving the structure of each underlying form in order to be human readable – as shown in Figure 1. Modeling these stylistic attributes and how they compose with underlying character structure could aid typographic analysis and even allow for auto- matic generation of novel fonts. Further, the vari- ability of these stylistic features presents a chal- lenge for optical character recognition systems, which typically presume a library of known fonts. In the case of historical document recognition, for example, this problem is more pronounced due to the wide range of lost, ancestral fonts present in such data (Berg-Kirkpatrick et al., 2013; Berg- Kirkpatrick and Klein, 2014). Models that capture this wide stylistic variation of glyph images may eventually be useful for improving optical charac- ter recognition on unknown fonts. In this work we present a probabilistic latent variable model capable of disentangling stylistic features of fonts from the underlying structure of each character. Our model represents the style of each font as a vector-valued latent variable, and parameterizes the structure of each character as a learned embedding. Critically, each style latent variable is shared by all characters within a font, while character embeddings are shared by char- acters of the same type across all fonts. Thus, our approach is related to a long literature on us- ing tensor factorization as a method for disentan- gling style and content (Freeman and Tenenbaum, 1997; Tenenbaum and Freeman, 2000; Vasilescu and Terzopoulos, 2002; Tang et al., 2013) and to recent deep tensor factorization techniques (Xue et al., 2017). Inspired by neural methods’ ability to disentan- gle loosely coupled phenomena in other …

Content truncated to 500 words