The Power of Sound Design in a Moving Picture: an Empirical Study with emoTouch for iPad

Abstract

The art of sound design for a moving picture rests basically on the work experience of pragmatists. This study tries to establish some guidelines on sound design: In an experiment 240 participants gave feedback about their emotions while watching two videos, each combined with four different audio tracks – music, sound effects, full sound design (music and sound effects) and no audio (as the comparative "null" version). Each participant viewed an audiovisual combination once to prevent habituation. The lead author employed a tablet-computer with the emoTouch-application serving as a mapping tool to provide information about the emotional responses. The participants moved a marker on the tablet's touch screen in a two-dimensional rating scale describing their felt immersion and suspense. A 3-factor-ANOVA showed significant increases of the median (and maximum) values of immersion and suspense when the participants listened to music and/ or sound effects. These values were always compared to the induced emotions of the participants who watched the videos with no audio at all. The video with full sound design audio tracks increased the median immersion values up to four times and the median suspense values up to 1.4 times. The median suspense values of the video with either music or sound effects dropped by 40 percent compared to the median suspense values of the null version. In contrast, the median immersion values were increased up to 3.6 times. The findings point to the importance of sound effects in an appropriate mix with music to enhance the viewers induced immersion and suspense.