A Case Study of Deep Learning-Based Multi-Modal Methods for Labeling the Presence of Questionable Content in Movie Trailers

Proceedings of Recent Advances in Natural Language Processing, pages 1297–1307 Sep 1–3, 2021. https://doi.org/10.26615/978-954-452-072-4_146 1297 A Case Study of Deep Learning-Based Multi-Modal Methods for Labeling the Presence of Questionable Content in Movie Trailers Mahsa Shafaei* , Christos Smailis*, Ioannis A. Kakadiaris, Thamar Solorio Department of Computer Science, University of Houston, Houston, TX, USA mshafaei@uh.edu, csmailis@central.uh.edu, ikakadia@central.uh.edu, tsolorio@uh.edu * These authors contributed equally to this work. Abstract In this work, we explore different approaches to combine modalities for the problem of au- tomated age-suitability rating of movie trail- ers. First, we introduce a new dataset contain- ing videos of movie trailers in English down- loaded from IMDB and YouTube, along with their corresponding age-suitability rating la- bels. Secondly, we propose a multi-modal deep learning pipeline addressing the movie trailer age suitability rating problem. This is the first attempt to combine video, audio, and speech information for this problem, and our experimental results show that multi-modal approaches significantly outperform the best mono and bimodal models in this task. 1 Introduction Movie trailers can be found in abundance through- out the web using services such as video streaming platforms. However, not all types of content in trailers are suitable for every audience. Specifically, movie trailers may contain explicit, aggressive, or violent content that may be harmful to the psyche of young viewers. Previous research has documented that some of the negative effects of mass media in young viewers include aggression and anxiety (Wilson, 2008; Chang and Bushman, 2019), as well as increasing the risk of sexual onset and alcohol and drug consumption, unwanted pregnancies, and sexually transmitted diseases (Strasburger, 1989). The Advertising Administration of the Motion Picture Association of America (“MPAA”) es- tablished guidelines for manually rating the age- suitability of movie trailers (Motion Picture Associ- ation). The rating of movie trailers is independent of the rating of the movie itself, as a trailer in- cludes only a short overview of the entire movie. Due to the time consuming nature, as well as the challenges to scale the MPAA rating process, au- tomating the task is of practical value. Moreover, automating the task poses interesting challenges to multi-modal classification systems, as the source of the objectionable content can come from any, or the combination of, these sources: language (use of bad words or discussion of adult themes), images (graphic violent scenes, nudity, drug or alcohol use), and audio (loud noises and music score de- noting suspenseful content). A successful rating approach should integrate evidence provided by the multiple modalities when making the predictions. In this paper, we study the performance of differ- ent multi-modal deep learning methods, to automat- ically predict the MPAA age-suitability rating of movie trailers using cues from the video, audio, and text modalities. Our goal is to show the feasibility of automating the rating task, and in particular, the relevance of multimodal solutions. We explore the use of late fusion, feature concatenation fusion and Gated Multi-modal Unit (GMU) Fusion (Arevalo et al., 2017). Since the proposed pipeline does not …

Content truncated to 500 words