Refining Data for Text Generation

Refining Data for Text Generation Qianying Liu12, Tianyi Li1, Wenyu Guan1 and Sujian Li1 1 Key Laboratory of Computational Linguistics, MOE, Peking University 2 Graduate School of Informatics, Kyoto University ying@nlp.ist.i.kyoto-u.ac.jp; litianyi01@pku.edu.cn ; guanwy@pku.edu.cn; lisujian@pku.edu.cn Abstract Recent work on data-to-text generation has made progress under the neural encoder-decoder architectures. However, the data input size is often enormous, while not all data records are important for text generation and inappropriate input may bring noise into the final output. To solve this problem, we propose a two-step approach which first selects and orders the important data records and then generates text from the noise-reduced data. Here we propose a learning to rank model to rank the importance of each record which is supervised by a relation extractor. With the noise-reduced data as input, we implement a text generator which sequentially models the input data records and emits a summary. Experiments on the ROTOWIRE dataset verifies the effectiveness of our proposed method in both performance and efficiency. 1 Introduction Recently the task of generating text based on structured data has attracted a lot of interest from the natural language processing community. In its early stage, text generation (TG) is mainly accomplished with manually compiled rules or templates, which are inflexible and mainly based on expert knowledge (Kukich, 1983; Holmes-Higgin, 1994; Reiter and Dale, 1997). With the development of neural network techniques, especially sequence-to-sequence (seq2seq) models, generating short descriptive texts from structured data has achieved great successes, including generating wikipedia-style biographies (Lebret et al., 2016; Sha et al., 2017) and restaurant introductions (Novikova et al., 2017). However, the task of generating long text, such as generating sports news from data, still fails to achieve satisfactory results. The existing models often forge fake context, lose sight of key facts and display inter-sentence incoherence (Wiseman et al., 2017). For the sports news generation task, one challenging problem is that the input records are both large and noisy. Specifically, the inputted box scores, which contains hundreds of data records, belong to 40 different categories, such as fouls, three- pointer, starting position and so on. Meanwhile, not all of the inputted records are reflected in the sports news, and there exists a serious non-parallelism between data records and texts. According to our statistics for 3000 parallel sports news and its data records which is shown in Table 1 and Figure 1, an average of only 19.3 data records out of 670.6 are mentioned in the summaries on average, namely only less than 5% of the data records are reflected in the human written news and rest 95% of them may bring noise into the model. Such large and noisy input has also caused the parameter amount of the embedding and encoder layer to be enormous, which leads to massive memory usage and limits the computation speed. In such situation, it is essential to refine data records and choose those important information before generating the final text. In addition, sport news is far more complex than short descriptive text …

Content truncated to 500 words