A Joint Model for Semantic Sequences: Frames, Entities, Sentiments

Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017), pages 173–183, Vancouver, Canada, August 3 - August 4, 2017. c⃝2017 Association for Computational Linguistics A Joint Model for Semantic Sequences: Frames, Entities, Sentiments Haoruo Peng Snigdha Chaturvedi Dan Roth University of Illinois, Urbana-Champaign {hpeng7,snigdha,danr}@illinois.edu Abstract Understanding stories – sequences of events – is a crucial yet challenging nat- ural language understanding task. These events typically carry multiple aspects of semantics including actions, entities and emotions. Not only does each individ- ual aspect contribute to the meaning of the story, so does the interaction among these aspects. Building on this intuition, we pro- pose to jointly model important aspects of semantic knowledge – frames, entities and sentiments – via a semantic language model. We achieve this by first represent- ing these aspects’ semantic units at an ap- propriate level of abstraction and then us- ing the resulting vector representations for each semantic aspect to learn a joint rep- resentation via a neural language model. We show that the joint semantic language model is of high quality and can gener- ate better semantic sequences than models that operate on the word level. We further demonstrate that our joint model can be applied to story cloze test and shallow dis- course parsing tasks with improved perfor- mance and that each semantic aspect con- tributes to the model. 1 Introduction Understanding a story requires understanding se- quences of events. It is thus vital to model se- mantic sequences in text. This modeling process necessitates deep semantic knowledge about what can happen next. Since events involve actions, participants and emotions, semantic knowledge about these aspects must be captured and modeled. Consider the examples in Figure 1. In Ex.1, we observe a sequence of actions (commit, arrest, charge, try), each corresponding to a predicate Ex.1 (Actions - Frames) Steven Avery committed murder. He was arrested, charged and tried. Opt.1 Steven Avery was convicted of murder. Opt.2 Steven went to the movies with friends. Alter. Steven was held in jail during his trial. Ex.2 (Participants - Entities) It was my first time ever playing football and I was so nervous. During the game, I got tackled and it did not hurt at all! Opt.1 I then felt more confident playing football. Opt.2 I realized playing baseball was a lot of fun. Alter. However, I still love baseball more. Ex.3 (Emotions - Sentiments) Joe wanted to be- come a professional plumber. So, he applied to a trade school. Fortunately, he was accepted. Opt.1 It made Joe very happy. Opt.2 It made Joe very sad. Alter. However, Joe decided not to enroll because he did not have enough money to pay tuition. Figure 1: Examples of short stories requiring different aspects of semantic knowledge. For all stories, Opt.1 is the correct follow-up, while Opt.2 is the contrastive wrong follow-up demonstrating the importance of each aspect. Alter. showcases an alternative correct follow-up, which requires considering different aspects of semantics jointly. frame. Clearly, “convict” is more likely than …

Content truncated to 500 words