Learning to generate one-sentence biographies from Wikidata

Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pages 633–642, Valencia, Spain, April 3-7, 2017. c⃝2017 Association for Computational Linguistics Learning to generate one-sentence biographies from Wikidata Andrew Chisholm University of Sydney Sydney, Australia andy.chisholm.89@gmail.com Will Radford Hugo Australia Sydney, Australia wradford@hugo.ai Ben Hachey Hugo Australia Sydney, Australia bhachey@hugo.ai Abstract We investigate the generation of one- sentence Wikipedia biographies from facts derived from Wikidata slot-value pairs. We train a recurrent neural network sequence-to-sequence model with atten- tion to select facts and generate textual summaries. Our model incorporates a novel secondary objective that helps en- sure it generates sentences that contain the input facts. The model achieves a BLEU score of 41, improving significantly upon the vanilla sequence-to-sequence model and scoring roughly twice that of a sim- ple template baseline. Human preference evaluation suggests the model is nearly as good as the Wikipedia reference. Manual analysis explores content selection, sug- gesting the model can trade the ability to infer knowledge against the risk of hallu- cinating incorrect information. 1 Introduction Despite massive effort, Wikipedia and other col- laborative knowledge bases (KBs) have coverage and quality problems. Popular topics are covered in great detail, but there is a long tail of special- ist topics with little or no text. Other text can be incorrect, whether by accident or vandalism. We report on the task of generating textual sum- maries for people, mapping slot-value facts to one- sentence encyclopaedic biographies. In addition to initialising stub articles with only structured data, the resulting model could be used to improve con- sistency and accuracy of existing articles. Figure 1 shows a Wikidata entry for Mathias Tuomi, with fact keys and values flattened into a sequence, and the first sentence from his Wikipedia article. Some values are in the text, others are missing TITLE mathias tuomi SEX OR GENDER male DATE OF BIRTH 1985-09-03 OCCUPATION squash player CITIZENSHIP finland Figure 1: Example Wikidata facts encoded as a flat input string. The first sentence of the Wikipedia article reads: Mathias Tuomi, (born September 30, 1985 in Espoo) is a professional squash player who represents Finland. (e.g. male) or expressed differently (e.g. dates). We treat this knowlege-to-text task like trans- lation, using a recurrent neural network (RNN) sequence-to-sequence model (Sutskever et al., 2014) that learns to select and realise the most salient facts as text. This includes an attention mechanism to focus generation on specific facts, a shared vocabulary over input and output, and a multi-task autoencoding objective for the com- plementary extraction task. We create a reference dataset comprising more than 400,000 knowledge- text pairs, handling the 15 most frequent slots. We also describe a simple template baseline for com- parison on BLEU and crowd-sourced human pref- erence judgements over a heldout TEST set. Our model obtains a BLEU score of 41.0, com- pared to 33.1 without the autoencoder and 21.1 for the template baseline. In a crowdsourced preference evaluation, the model outperforms the baseline and …

Content truncated to 500 words