Amplifying a Sense of Emotion toward Drama-Long Short-Term Memory Recurrent Neural Network for dynamic emotion recognition

Amplifying a Sense of Emotion toward Drama- Long Short-Term Memory Recurrent Neural Network for dynamic emotion recognition Huang-Cheng Chou Department of Electrical Engineering National Tsing Hua University hc.chou@gapp.nthu.edu.tw Chun-Min Chang Department of Electrical Engineering National Tsing Hua University cmchang@gapp.nthu.edu.tw Yu-Shuo Liu Department of Electrical Engineering National Tsing Hua University drugstore950337@gmail.com Shiuan-Kai Kao Department of Electrical Engineering National Tsing Hua University yu21a90v30l60@gmail.com Chi-Chun Lee Department of Electrical Engineering National Tsing Hua University cclee@ee.nthu.edu.tw 摘要 本篇論文探討放大觀眾欣賞戲劇時候的情緒，使用 Long Short-Term Memory Recurrent Neural Network 來當作主要的模型，建立和預測連續動態的情緒。建立好 模型後，我們將結果移植到視覺程式上，我們可以達成兩個不同的情緒視覺放大器， 分別有 RGB 和 Vignette 版本，RGB 版本是控制影像中的 RGB 像素數值，及時隨 著模型結果 (Arousal 和 Valence 值)，及時調整紅色、綠色和藍色數值，讓觀眾隨著影像中演員的情 緒起伏；同樣也是，我們視覺化換成 Vignette，隨著情緒辨識維度(Arousal 和 Valence)， The 2017 Conference on Computational Linguistics and Speech Processing ROCLING 2017, pp. 136-147  The Association for Computational Linguistics and Chinese Language Processing 136 來產生情緒視覺化畫面。而我們使用資料庫是 NNIME，NNIME 是最近收集的多模態 互動情緒資料庫，總共有 44 位演員，約 11 個小時的連續資料，包含語音、影像和生 理訊號，這個嚴謹和工程化收集的資料庫，提供給我們很好研究情緒的機會，讓我們 得以完成此篇研究。 Abstract This paper tried to amplify a sense of emotion toward drama. Using Long Short-Term Memory Recurrent Neural Network to model and predict dynamic emotion(Arousal and Valence) recognition. After building model, we transplant whole framework and take results from it on visualizing. We have two demo version: RGB version and Vignette version. RGB version is to modulate the RGB value of frame in video. The Vignette one is to add the vignette effect. Both version all are to amplify a sense of emotion toward drama. Let people have more fun during watching videos. The database we used is NNIME (The NTHU-NTUA Chinese Interactive Multimodal Emotion Corpus) [1]. NNIME is a newly- collected multimodal corpus. This database includes recordings of 44 subjects engaged in spontaneous dyadic spoken interaction. The length of data is about 11 hours containing audio, video and electrocardiogram. The database is also completed with a rich set of emotion annotations on continuous-in-time annotation by four annotators. This carefully- engineered data collection and annotation processes provide us to create amplify framework. Keywords: Affective computing, emotion recognition, Long Short-Term Memory, Recurrent Neural Network 1. Introduction Emotion is a core fundamental internal attribute of humans that governs our behaviors and decision-makings. There has already been a tremendous research effort in modeling humans using a variety of measurable signals, which aims at enabling machines to sense emotional states automatically [2, 3]. One key components in advancing such research is the availability of databases for researchers to develop robust recognition algorithms and carry out meaningful analyses. Human interaction 137 often involves complex processes of communicative goals and emotional behaviors, which not only are expressed verbally and nonverbally but also are reflected in the inner responses of humans [4]. We want to use the NNIME database to study the emotion behavior (such as arousal and valence state) in small duration (like in real time), and to augment a sense of emotional feeling with visual demonstration. After all of this, we just wonder how the emotion application can be. Therefore, we think of the amplification of the emotion in video. There are a lot of video that you would feel awkward watching it because of its boring and no effect. Moreover, we …

Content truncated to 500 words