Scalable Font Reconstruction with Dual Latent Manifolds

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 3060–3072 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 3060 Scalable Font Reconstruction with Dual Latent Manifolds Nikita Srivatsan Language Technologies Institute Carnegie Mellon University nsrivats@cmu.edu Si Wu Khoury College of Computer Science Northeastern University siwu@ccs.neu.edu Jonathan T. Barron Google Research barron@google.com Taylor Berg-Kirkpatrick Computer Science and Engineering University of California, San Diego tberg@eng.ucsd.edu Abstract We propose a deep generative model that performs typography analysis and font re- construction by learning disentangled man- ifolds of both font style and character shape. Our approach enables us to mas- sively scale up the number of character types we can effectively model compared to previous methods. Specifically, we infer separate latent variables representing char- acter and font via a pair of inference net- works which take as input sets of glyphs that either all share a character type, or be- long to the same font. This design allows our model to generalize to characters that were not observed during training time, an important task in light of the relative spar- sity of most fonts. We also put forward a new loss, adapted from prior work that measures likelihood using an adaptive dis- tribution in a projected space, resulting in more natural images without requiring a discriminator. We evaluate on the task of font reconstruction over various datasets representing character types of many lan- guages, and compare favorably to modern style transfer systems according to both au- tomatic and manually-evaluated metrics. 1 Introduction The majority of written natural language comes to us in the form of glyphs, visual repre- sentations of characters generally rendered in a font with a contextually appropriate style. In order to be legible a glyph must be recog- nizable as its corresponding character from its underlying shape, but it must also be stylisti- cally consistent with the other glyphs in that font. While this labor intensive process is typ- ically performed manually by human artists, Font Character Glyph Figure 1: Supported glyphs in Google Fonts orga- nized by character type and font. A blue pixel in- dicates that column’s font includes that row’s char- acter. Our proposed model allows font reconstruc- tion over this large, sparse character set. the number of character types that a font may be expected to support is extremely large, with Unicode 13.0.0 including as many as 143,859 character types (Unicode). As a re- sult, graphic designers often create glyphs only for a subset of these characters, which tends to be determined by their own cultural context. This can create an accessibility gap for users seeking to create or read digital content in languages with less widespread orthographies, due to the relative lack of available options. Figure 1 shows that within the Google Fonts library (Google) there is a long tail of fonts with a large proportion of missing glyphs. Font reconstruction is a task that attempts to solve this problem. The goal is for a model, given a small set of example …

Content truncated to 500 words