Handwritten Character Generation using Y-Autoencoder for Character Recognition Model Training

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 7344–7351 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 7344 Handwritten Character Generation using Y-Autoencoder for Character Recognition Model Training Tomoki Kitagawa1, Chee Siang Leow1,2, Hiromitsu Nishizaki1 1Integrated Graduate School of Medicine, Engineering, and Agricultural Sciences, University of Yamanashi 4-3-11 Takeda, Kofu, Yamanashi 400-8511, JAPAN 2Artibrains LLC, 3-8-6 Joto, Kofu, Yamanashi 400-0861, JAPAN kitagawatomoki@alps-lab.org, siang@artibrains.com, hnishi@yamanashi.ac.jp Abstract It is well-known that the deep learning-based optical character recognition (OCR) system needs a large amount of data to train a high-performance character recognizer. However, it is costly to collect a large amount of realistic handwritten characters. This paper introduces a Y-Autoencoder (Y-AE)-based handwritten character generator to generate multiple Japanese Hiragana characters with a single image to increase the amount of data for training a handwritten character recognizer. The adaptive instance normalization (AdaIN) layer allows the generator to be trained and generate handwritten character images without paired-character image labels. The experiment showed that the Y-AE could generate Japanese character images then used to train the handwritten character recognizer, producing an F1-score improved from 0.8664 to 0.9281. We further analyzed the usefulness of the Y-AE-based generator with shape images and out-of-character (OOC) images, which have different character image styles in model training. The result showed that the generator could generate a handwritten image with a similar style to that of the input character. Keywords: handwritten characters generation, optical character recognition (OCR), Y-Autoencoder 1. Introduction The re-emergence of deep learning since the third winter of artificial intelligence has led to significant achievements in various fields. Specifically, it has led to the mainstreaming of deep learning-based sys- tems that use large amounts of data to train a model. Supervised-learning deep learning models have been particularly successful among the various deep learn- ing methods. Various studies have extensively in- vestigated semi-supervised and unsupervised learn- ing; however, most commercialized systems still com- monly use supervised learning. In the fields of text image recognition among the vari- ous research fields, machine-printed optical character recognition (OCR) (Li et al., 2019; Min et al., 2020), and connectionist temporal classification (CTC)-based (Graves et al., 2006) end-to-end handwritten text recognition (HTR) (Graves and Schmidhuber, 2008; Ly et al., 2017; Ingle et al., 2019) require a large amount of data to train, which makes the text rec- ognizer more generalized and high-performance. Therefore, data augmentation, such as CutMix (Yun et al., 2019), mixup (Zhang et al., 2018), and Randaugment (Cubuk et al., 2020), and mosaic aug- mentation (Hao and Zhili, 2020), which is based on image processing, is generally used to learn more general models by increasing the amount of data. However, adequate data augmentation methods still have to be designed by humans. Therefore, sev- eral methods of increasing the amount of data for supervised learning-based deep learning models re- quire a large amount of data, including online-based character generation using recurrent neural networks (RNNs) and offline-based adversarial adaptive gen- eration networks (GANs), which generate images. …

Content truncated to 500 words