Keeping Consistency of Sentence Generation and Document Classification with Multi-Task Learning

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 3195–3205, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 3195 Keeping Consistency of Sentence Generation and Document Classification with Multi-Task Learning Toru Nishino Tomoki Taniguchi Shotaro Misawa Yasuhide Miura Fuji Xerox Co., Ltd. {nishino.toru, misawa.shotaro, kano.ryuji, taniguchi.tomoki, yasuhide.miura, ohkuma.tomoko} @fujixerox.co.jp Ryuji Kano Tomoko Ohkuma Abstract The automated generation of information in- dicating the characteristics of articles such as headlines, key phrases, summaries and categories helps writers to alleviate their work- load. Previous research has tackled these tasks using neural abstractive summarization and classification methods. However, the outputs may be inconsistent if they are generated individually. The purpose of our study is to generate multiple outputs consistently. We introduce a multi-task learning model with a shared encoder and multiple decoders for each task. We propose a novel loss function called hierarchical consistency loss to main- tain consistency among the attention weights of the decoders. To evaluate the consistency, we employ a human evaluation. The results show that our model generates more consistent headlines, key phrases and categories. In addition, our model outperforms the baseline model on the ROUGE scores, and generates more adequate and fluent headlines. 1 Introduction Headlines and other information such as key phrases, summaries and categories about articles are crucial for readers to search articles on demand. To attract more readers, writers manually create headlines and summaries by summarizing the articles, extract key phrases and classify articles into categories. Figure 1 shows an example of job advertisement articles. In addition to the job description text, the headline, key phrase and category are labeled for each article. Thus job seekers can easily retrieve the job advertisements they desire by reading them. However, it is a burden for writers to create these headlines, key phrases and categories manually for extremely large numbers of articles. Hence, an automatic generation system is highly demanded. Headline: We Want Android1 Engineer to Develop our New Service “Wantedly People2 ”! Key Phrase: Android Engineer Category: Engineer Description Text (Truncated): We released a new service ”Wantedly People” in November 2016, and released new features in July 2017! Our “People team”, developing rapidly growing “Wantedly People” service, is recruiting an Android Engineer! If you love to work on new services, or if you want to try to develop new apps as your representative work, we are looking forward to your application! Figure 1: An example job advertisement article with consistency. Each article contains a headline, key phrase and category. The underlined topic “Engineer” is consistently noted in the headline, key phrase and category. A Japanese-English translation is applied. For automated generation of multiple outputs, consistency among outputs is crucial. A lack of consistency among outputs causes incorrect infor- mation in the outputs. In Figure 1, for example, the “Engineer” position is consistently noted in the headline, key phrase and category. An occupation is consistent and salient information for headlines, key phrases …

Content truncated to 500 words