THE paper "The Power of Sound Design in a Moving Picture. An Empirical Study with emoTouch for iPad" by Kock and Louven is a valuable attempt to provide insight into the thus far neglected research area on the effects and status of sound in film. In a 2x2 between-subjects design, which varied the factors sound/ no sound and music/ no music in an animated and a realistic film stimulus, 240 participants rated their perceived suspense and immersion continuously while watching a film stimulus using the emoTouch software. Afterward, potential intervening variables such as the participants' music and film expertise were measured. The authors described particularly strong effects regarding the participants' immersion and perceived suspense in the experimental conditions with complete sound design (combination of music and sound) compared to the versions without audio. Therefore, Kock and Louven underlined the impact of sound on the effects of films. They also indicated an overestimation of the immersive and suspenseful power of film music by itself.

METHODOLOGICAL CONSIDERATIONS

Responding to a stimulus in real time – especially if it is measured on two different dimensions – claims large cognitive resources, and, therefore, could impair the quality of responses. To establish some guidelines on the importance of sound in a film – an independent variable that unfolds in time – and given the current literature, which suggests meaningful and reasonably consistent results concerning this method (Schubert, 2010, p. 241), a continuous self-report seems reasonable. In addition, the authors gave their participants the opportunity to practice with the software before starting the experiment. Nevertheless, Kock and Louven themselves pointed out that, for example, a strong immersion of the participants could have led to inaccurate handling of the software and thus to falsified measurements. Since the conditions for all participants were the same due to the experimental set-up, the results are at least internally comparable. However, the authors did not use the full potential of their very elaborate data by interpreting the progression of the data in detail (e.g., a comparison of the characteristics of the individual conditions at certain relevant points in the film plots or the interpretation of specific maxima or minima).

The dependent variables of suspense and immersion have already been proven to be relevant in other experiments on sound in film (e.g., Costabile & Terman, 2013; Tan, Spackman, & Wakefield, 2017). In terms of a continuous measure of suspense, however, in future studies, a distinction between felt and perceived emotions could be considered (Lehne & Koelsch, 2015, p. 3). Based on the possibly existing expert knowledge of this study's student sample the participants could have indicated scenes as suspenseful that were meant to trigger suspense, without actually feeling it.

Kock and Louven mentioned a balance of immersion in the music-only condition of both film types in their study. However, an important question remained unanswered: Is a stimulus with varying degrees of suspense maybe perceived as more suspenseful overall than a stimulus with consistently high suspense because of habituation effects? Or vice versa – Does delaying the resolution of tension increase the tension experience (de Wied, 1995)? Future studies may regard an additional measurement of post-treatment suspense.

Some theories and studies suggest the structural correspondence of audible and visual elements as relevant to the effect of film music. For example, music can mimic movements of the filmed events (e.g., footsteps, pendular movement of a big clock) or accentuate the editing of the film scenes (Bullerjahn, 2005). Marshall and Cohen (1988) found in their study a counterintuitive influence of music on the perception of the film characters. One possible explanation was an unintentional accentuation of one of the film characters by the musical structure. On the other hand, Bolivar, Cohen, and Fentress (1994) found no influence of the temporal congruence of music and film. Therefore, in recent film music studies structural correspondence is often considered (Costabile & Terman, 2013; Tan, Spackman, & Bezdek, 2007). Structural congruence can be created or deliberately avoided not only between music and filmed events but also within the auditory levels (music and sounds) – and could be investigated in future experiments.

Music-focused studies show that the degree of realism of the stimuli can have a major impact on the effect of the audible elements (e.g., Cohen, 1993; Thompson, Russo, & Sinclair, 1994). Accordingly, it seems natural – as proposed by Kock and Louven – to investigate this factor in terms of sound (which obviously can contribute to the degree of realism of a stimulus more than music). Therefore, this study used two meticulously produced film types differing in their degree of realism: a computer-generated animatic movie and a live-action movie. However, one possible limitation could be that the films differed not only in type, but also in their plot, perceived closure, in the music and sounds used, and in their duration. The specific effects of the soundtrack in the animated and the realistic film can not necessarily be attributed to the different film types.

RELEVANCE OF THE RESEARCH APPROACH

"Within a film, the soundtrack contains not only the musical score, but ambient sound, dialogue, sound effects, and silence […]. We would argue that the composite mix of these sounds becomes a complex communicative form that could be considered – in toto – 'music' (i.e., ordered sound), existing 'in harmony' with the visual image. […] We suggest, however, that as the field [of film music research] continues to mature, the constituent elements that comprise the soundtrack should be studied as a whole" (Lipscomb & Tolchinsky, 2005, pp. 401-402).

In 2005, Lipscomb and Tolchinsky already demanded not to isolate music as a single factor in the research of film music effects, but to map the entire spectrum of film sound in experiments. So far, some studies have recognized that the experimental logic requires a control condition with sounds (e.g., Tan et al., 2007, p. 139) because completely silent stimuli appear too unrealistic (Chion, 1994). However, a focus on the film sound as an influential film parameter is currently not noticeable. Therefore, this study marks a further step towards the development of a more comprehensive insight into the effects of sound design in film.

This study's results do not only suggest that a specific combination of music and sound leads to strong effects in terms of the participants' perceived immersion and tension. The range of these results also indicates that under certain circumstances music alone (immersion in animated film, Fig. 5), sounds alone (immersion in real film, Fig. 6) or even silence (suspense in both film conditions, Fig. 7 and 8) can have particularly intense effects. The necessity mentioned by the authors to establish experimental, as well as pragmatically-founded guidelines for the use of sound design in film, is underlined by these unexpected findings and, hopefully, will be the impetus for research that can establish scientific guidelines with practical implications, too.