PS-GAN: Feature augmented text generation in Telugu

PS-GAN: Feature augmented text generation in Telugu Pratyusha Musunuru∗ Blaize pratyusha.musunuru@blaize.com Varshit Battu∗ Plivo varshit.battu@plivo.com Abstract Since the inception of Generative Adver- sarial Networks (GANs), synthetic im- age generation has taken a giant leap because of the ability of these networks to generate high-quality images, however, the same cannot be said for text genera- tion. A major challenge encountered in text generation using GAN’s is the non- differentiability of the discrete text. Most of the previous studies for text genera- tion using GANs focus on solving this, but none of them incorporate any addi- tional features in the GAN. These fea- tures could be useful in the training of the models, especially in the case of low- resource languages. In this paper, we pro- pose a novel model called the POS-Senti- GAN (PS-GAN), where we show that the use of Parts-Of-Speech tag and sentiment features aid in the generation of better sentences. We also provide ‘Pravar’, a first-ever dataset consisting of stories from different categories that enable text/story generation for Telugu, a low resource lan- guage. Finally, we show the performance of the proposed models on three datasets, namely, Pravar1, Telugu Wikipedia2 and Telugu News3. ∗ The authors contributed equally to this work. 1Pravar is a Hindi word that translates to “good” or “important”. 2https://www.kaggle.com/disisbig/telugu-wikipedia- articles 3https://github.com/AnushaMotamarri/Telugu- Newspaper-Article-Dataset 1 Introduction Text generation strives towards generating text close to human written text by leveraging syn- tactic, semantic knowledge and artificial intelli- gence in the formation of sentences. This field of text generation can be applied to many natural language processing tasks such as generating re- sponses to users’ queries, translating a sentence or a document from one language into another, offering suggestions to help write a story, gen- erating summaries, etc. GANs were first introduced by (Goodfellow et al., 2014). They learn by solving the min- max optimization problem, i.e., the discrimina- tor seeks to maximize the likelihood for the ac- tual data to be identified as real and the gener- ated data to be identified as fake, whereas the generator seeks to minimize the likelihood for its generated data to be identified as fake. When using GANs on text, we run into issues while back-propagating the discriminator loss to the generator. As mentioned in (Iqbal and Qureshi, 2020), differentiating a pixel of an image makes sense, whereas differentiating a word (discrete stochastic unit) doesn’t. This problem was ad- dressed in two ways, (Jang et al., 2016) used an efficient gradient estimator that replaces the non-differentiable sample from a categorical dis- tribution with a differentiable sample from a novel Gumbel-Softmax distribution, and (Yu et al., 2017) used reinforcement learning. Out of the many features present in the text, Parts-Of-Speech(POS) tags and sentiment are the most prominent ones. POS tags carry the semantic features of a sentence. They are a crucial part of every language as they serve as building blocks for its grammar, particularly in the linguistic domain. POS tags are an essen- tial feature used in multiple fields like informa- …

Content truncated to 500 words