StoryWars: A Dataset and Instruction Tuning Baselines for Collaborative Story Understanding and Generation

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 3044–3062 July 9-14, 2023 ©2023 Association for Computational Linguistics STORYWARS: A Dataset and Instruction Tuning Baselines for Collaborative Story Understanding and Generation Yulun Du and Lydia Chilton Columbia University New York City, New York, USA {yulundu, chilton}@cs.columbia.edu Abstract Collaborative stories, which are texts created through the collaborative efforts of multiple authors with different writing styles and inten- tions, pose unique challenges for NLP models. Understanding and generating such stories re- mains an underexplored area due to the lack of open-domain corpora. To address this, we introduce STORYWARS, a new dataset of over 40,000 collaborative stories written by 9,400 different authors from an online platform. We design 12 task types, comprising 7 understand- ing and 5 generation task types, on STORY- WARS, deriving 101 diverse story-related tasks in total as a multi-task benchmark covering all fully-supervised, few-shot, and zero-shot sce- narios. Furthermore, we present our instruction- tuned model, INSTRUCTSTORY, for the story tasks showing that instruction tuning, in addi- tion to achieving superior results in zero-shot and few-shot scenarios, can also obtain the best performance on the fully-supervised tasks in STORYWARS, establishing strong multi-task benchmark performances on STORYWARS.1 1 Introduction Storytelling is crucial due to its vital role in hu- man experience, history, and culture dating back to the earliest days of humanity. Humans possess the unique storytelling ability to structure a sequence of events, whether factual, fictional or a mixture of both, and create a coherent narrative that conveys a big picture while also including intricate details. Current story generation systems usually mimic this ability by starting with a plot then crafting the story. This can be done by linearly expand- ing (Peng et al., 2018, Yao et al., 2019, Martin et al., 2017) or hierarchically developing (Xu et al., 2018, Fan et al., 2018, Fan et al., 2019, Rashkin et al. 2020, Goldfarb-Tarrant et al., 2020) the story based on the given plot. Collaborative storytelling 1We make our data, code, and models publicly available at https://github.com/ylndu/storywars These are a series of letters found in my basement. Not sure if this is a prank or something but here they are, a warning though, it's pretty weird. Letter #1 Greetings! You might not know me but I know you. I've been watching you. All the decisions you made (even the bad ones) and all the roads you took to get here, but all good things must come to an end. So, I'm here to wake you up! It's kind of the opposite of dying, in fact, you get to live again! Only. you won't remember a thing. The friends you made, the people you loved, the treasures you acquired over your lifespan, all will be gone. Your life will be sort of the same? I guess? It's basically your life all over again, except for one thing: I will be making your decisions for you this time! How cool is that? Super cool right! …

Content truncated to 500 words