IF one thing has become abundantly clear from decades of research in music cognition, it is that musical experiences are highly personal. Although it is clear that a range of common (or predictable) responses to music can be identified, there is no assurance that two people will feel the same way about a piece of music, or even necessarily have the same experience at repeated listening. The internalized, automatic processes that drive these experiences are the focus of the current paper. When perceiving music, we can generally say that a waveform of sound — with no inherent meaning — is transformed into music once it is perceived and interpreted through processes driven by the unique listening biography of the listener. This may in turn trigger various memory associations, or lead to differences in directing attention to specific musical aspects that are most salient to that person. The way we make sense of music thus depends on how we relate what we hear to previous experience — models built up from exposure to a lifetime of music, created by the unique auditory environment in which an individual has existed. This previously encountered environment provides the statistical basis for what we expect from auditory signals around us, by tracking incoming information and placing it into our own unique context of what we have previously experienced, based on the incidence of occurrences of those experiences. In this way, the unique implicit representations we have of what is common (and thus expected) drive our unique response to music. It also follows that having a greater overlap in music exposure in your listening biography with other people, as for instance through growing up in the same place, or being born in the same year, will lead to more similar responses to the same stimuli than when fewer biographical details are shared.

In the following, I will argue that the implicit expectations that influence perception and action are driven by mental models, and that research into music imagery offers an informative look at these mental representations of music, as well as the mechanisms of active, engaged perception. I here use the term "mental model" to refer to an internal representation of a percept or action, as it is built up through experience and statistical learning. Furthermore, I pose that these representations are informed and modulated by the body and its in- and outputs (namely perception and actions), in line with how the brain functions with its many efferent and afferent connections, thus making a strong case that mental representations have embodied components. The recently asserted theoretical framework of predictive processing as a driving mechanism in the coupling of perception, cognition and action, which also fits well with how we think about the predictions that drive music processing, includes an account of imagery as highly related to active perception, further supporting these ideas. Here, I will first discuss some of the work looking at music imagination in relation to mental models and then turn to findings of the coupling of perception, cognition and action, closing with more general considerations and implications.

Imagery and Perception

Imagery has been a topic of investigation for a considerable time, initially from a philosophical perspective, and more recently within the fields of experimental psychology and cognitive neuroscience. A detailed overview of experimental findings of music imagery is given by Hubbard (2010), who concludes that auditory imagery preserves many structural and temporal features of the imagined stimulus, involves semantically interpreted information and expectancies, has depictive and descriptive components and is related to musical abilities and experience. Neuroscientific research has found imagery in various modalities (i.e. visual, auditory, or movement imagery) to overlap with modality-specific perception or action in the brain, in terms of anatomical locations as well as in the temporal signatures of the electrical brain activity. Although the precise involvement of different neural regions, and specifically the primary modality-specific brain regions, is not fully established (cf. Daselaar, Porat, Huijbers & Pennartz, 2010), overlap in modality-specific brain activity has been reported for visual (cf. Kosslyn, Ganis & Thompson 2001; Cichy, Heinzle, & Haynes, 2012), movement (Pfurtscheller, Brunner, Schlögl & Da Silva, 2006; Oosterhof, Tipper, & Downing, 2012), and music imagination (Halpern & Zatorre 1999; Herholz, Halpern, & Zatorre, 2012; Herholz, Lappe, Knief, & Pantev, 2008; Schaefer, Desain & Farquhar, 2013; Schaefer, Desain & Suppes, 2009; Schaefer, Vlek & Desain, 2011a).

In thinking about what this shared activation represents, the first priority is to establish agreement on what we mean by imagery, as this diverges considerably in the literature. While most imagery research involves the deliberate imagination of sounds (cf. Halpern 2001 and others), visual images (cf. Cichy et al., 2012; Reddy, Tsuchiya, & Serre, 2010) or movements (cf. Oosterhof et al., 2012; Pfurtscheller et al., 2006), phenomena such as spontaneously imagined music (also termed "earworms") (Bailes, 2007; Williamson et al., 2011), comparisons of perception with a representation held in working memory (cf. Navarro Cebrian & Janata, 2010a, 2010b), manipulating elements of a musical image in working memory (Zatorre, Halpern, & Bouffard, 2009), judging the orientation of an object or body part (cf. Parsons, 1987, and many others, but see also Viswanathan, Fritz, & Grafton, 2012), implicit action outcome (cf. Keller, Dalla Bella, & Koch, 2010; Keller & Koch, 2008) or pitch expectations (Janata & Paroo, 2006), mental simulation of future scenarios (cf. Schacter, Addis, & Buckner, 2007) and various other tasks have all been considered imagery, or at least claimed to be supported by imagery processes. In some cases, the difference in task leads authors to question whether imagery really occurred (see for instance Hubbard's (2010) review in which this potential problem is mentioned repeatedly for music imagery), whereas in several cases the use of imagery is simply inferred from the idea that some form of modality-specific representation is being used to perform a task, such as is the case for the hand-turning paradigms in which no deliberate imagery of movement is performed (cf. Vingerhoets, de Lange, Vandemaele, Deblaere & Achten, 2002). Although these processes surely have something in common, there are also clear differences, at least phenomenologically. When comparing and interpreting experimental outcomes, it is crucial to take the differences between these cognitive tasks into account, with implications for the overlap that is seen with perceptual processes.

Coupling of Perception, Cognition and Action

The idea that perception, cognition and action are intricately coupled, and as such should not be seen (or investigated) as independent functional systems, is well supported by decades of research in music cognition, even more so than in general psychology. Bidirectional modulating influences have been demonstrated experimentally between perception and cognition, between perception and action and between cognition and action. Of these, the coupling between perception and cognition is perhaps the most extensively researched and needs the least description, with as obvious examples the cognitive models that have been described for tonal or temporal structures (cf. Deutsch & Feroe, 1981; Krumhansl, 1979; Palmer & Krumhansl, 1990) but of course also including effects of attention on perception, the effects of perceptual conditions on memory and so on.

Action and perception have also been shown to bidirectionally influence each other; we know that movements are regularized when carried out to music (Leman et al., 2013; Thaut, Kenyon, Hurt, McIntosh, & Hoemberg, 2002), that exercising to music decreases perceived exertion (Bood, Nijssen, van der Kamp, & Roerdink, 2013; Karageorghis & Priest, 2012) and that action is preattentively modulated by what we perceive (Furuya & Soechting, 2010). Currently, the use of auditory cues in rehabilitation of disordered movement is an active area of research (cf. Schaefer, 2014a) that holds considerable promise for therapies based on the coupling between perception and action. Going in the other direction, movement is reported to enhance temporal acuity in auditory perception (Manning & Schutz, 2013; Su & Pöppel, 2012), and to modulate the perception of rhythmically ambiguous temporal patterns (Phillips-Silver & Trainor, 2005, 2008). Previous experience appears to be crucial to this coupling, such that in highly trained musicians, the association between perception and action is sufficiently strong that just listening to music activates specific motor regions in the brain that are related to how the music would be performed (cf. Bangert et al., 2006; Haueisen & Knosche, 2001).

Finally, findings are now also emerging of cognition and action directly impacting each other. Examples of cognition impacting movement are that mental imagery helps music performance and practice (Holmes, 2005; Parncutt, 2007), both auditory and movement imagination (or mental training) aid motor learning (Brown & Palmer, 2013), and that "cognitive" states can be translated into actions, such as emotional expression being recognizable in dance (Burger, Saarikallio, Luck, Thompson, & Toiviainen, 2013). The influence of cognition on action has also been demonstrated at the level of movement-related brain activations; moving to imagined music was found to lead to different activation patterns within the motor network of the brain than when identical movements were carried out without music imagery (Schaefer, Morcom, Roberts & Overy, 2014). Probably the least investigated effect direction is that of movement impacting cognition, but various findings have been reported, specifically in investigations addressing specific aspects of the theoretical framework of embodied cognition (for a description of different aspects, see Wilson, 2002). For instance, the way we move influences emotional appraisal of music (Maes & Leman, 2013), and outside the realm of music research the effects of exercise on cognition are gaining considerable interest, including general effects of physical fitness (Angevaren, Aufdemkampe, Verhaar, Aleman, & Vanhees, 2008; Eggermont, Milberg, Lipsitz, Scherder, & Leveille, 2009), as well as findings that exercising while learning new words is more effective than learning in a passive condition (Schmidt-Kassow, Kulka, Gunter, Rothermich, & Kotz, 2010).

On closer inspection of these findings we may find that, although in these studies the impact of one functional domain (as in cognition, action and perception) on another was investigated, a role of the third is often inherent to the mechanism under investigation, and processes that have been considered imagery by some authors (although probably not the effortful, sensory kind) are arguably at play in cases focusing on action-perception coupling. For instance, in the case of rhythmic movement impacting later perception and preference, where babies bounced to differing interpretations of an ambiguous rhythm then showed a preference for that interpretation (Phillips-Silver & Trainor, 2005), it is likely that the movement created an internal accenting structure that led to a stable mental model of one interpretation of the ambiguous rhythm, something that would not have happened without some form of a mental representation of that accent structure, leading to a specific percept. A spontaneously emerging subjective accenting structure, taking the shape of an imagined pattern in time, has been shown to be present in perception-related brain signatures even when actual accents were never presented (Brochard, Abecasis, Potter, Ragot & Drake, 2003), but can also be added to ambiguous stimuli in a deliberate manner (Schaefer et al., 2011a). Mental representations of time structures have been argued to be inherent to both actual and imagined actions, similarly to perception and sensory imagery of music (Schaefer, 2014b), possibly even sharing neural resources. Likewise, in the anticipatory processing inherent to tapping along with a rhythm as mentioned earlier, the mental models of the rhythm appear to be directly informed by afferent motor feedback from the finger, as is suggested by the finding in a small sample of deafferented patients, who do not receive haptic or proprioceptic feedback from their movements and did not show the commonly seen slightly early taps, but instead were tapping right on time (Stenneken, Prinz, Cole, Paillard, & Aschersleben, 2006). These results imply that the motor system is crucially involved in setting up mental representations of temporal structure, highlighting the multimodal nature of the representations of rhythm that unfold over time.

Discussion

In the above, I have aimed to bring together ideas and findings from experimental psychology, cognitive neuroscience and philosophy to clarify the role of imagery in perception and action. The main argument is built on the assertion that what is referred to as imagery in the literature actually covers a range of phenomena, sometimes leading to conflicting findings and confusion in their interpretation, although in all cases including some sort of internal representation. These phenomena of imagery may occur with or without awareness, and with varying degrees of deliberateness and effort. Clarification of terminology and delineation of different types of imagery may help here, and subtypes of imagery are described.

The most relevant types for the current discussion are sensory imagery and constructive imagery, of which I here suggest that the latter may be included in the former, as a vividly imagined stimulus comprises structural and temporal features that need to be perceptually organized. That is, if indeed the structural and temporal features of an imagined stimulus are preserved (cf. Hubbard, 2010), the relations between pitches, durations, and even sections of a musical piece are retained in the image, including the unconscious predictions created for the different hierarchical levels of the stimulus, which can all be considered a part of perceptual organization. The concept of constructive imagery thus not only offers an account of the overlap between sensory imagery and active perception that is reported in the neuroimaging literature, but also fits well with a theoretical framework of predictive processing as a unifying mechanism in perception, action and cognition. The generative models that are assumed in this framework are informed by previous perception as well as action, leading to representations that impact our interpretation of what we perceive, which also include action-related aspects. The tight coupling of perception, action and cognition implies that the representations we have are multimodal, in that one set or level of predictions may be how we cognitively process an incoming stimulus (such as judging the tempo of a rhythm), whereas a set of predictions concern the behavioral response (such as tapping your foot) is generated concurrently, but not independently. Although this account is not the only one in which cognition, perception and action are thought of as tightly coupled (see for instance Hurley, 2008), the role of prediction error, which is so salient in music processing, emphasizes the importance of internal models in this coupling. Moreover, the multi-leveled structure of music may be exquisitely suited to further investigate more detailed questions about how these models and predictions may be generated, and whether there are specific types of models and predictions that are more strongly interrelated (such as perhaps that expectations about where in time harmonic changes happen are related to expectations of where in time a downbeat will occur), while other types may exist quite independently (such as the expectation of where a downbeat at a certain time point happening independently of which instrumental timbres are expected).

With the current contribution I aim to address an aspect of the personalized experience of music, based on internal mental models that incorporate characteristics related to both perception and action, and are generated from previous experience. By considering these internal mental models, which organize perception and direct our attention to specific stimulus features, as a type of imagery, as some authors already do, tension between research findings utilizing very different experimental tasks can be resolved. Although fully explaining individual differences in musical experiences is beyond the aim of the current perspective, the core idea that an individual combination of previous experience contributes to basic mechanisms of perception and action through construction of mental models that drive predictions that are crucial to perceptual processes is indeed relevant to these differences. Secondly, I argue for the relevance of researching music imagery with the goal of understanding how these mental models drive our musical experiences, while making clear distinctions in what aspect of imagery is researched, and what can be concluded from the results. For instance, by clarifying the impact of previous exposure on implicit, and memory functions on explicit, deliberate imagery tasks 4, gaining a better understanding of individual differences in sensory imagery ability, and the role of imagery in expert skills, a lot may be learned about the role of automatic as well as deliberate mental simulation in cognition, perception and action.

In the context of the embodied music cognition, this account may still sound quite cognition-centered, however from a neurologist's perspective, the concept of the "non-neural body" or, similarly, the disembodied brain makes very little sense, given the rich bidirectional connections within the central nervous system as well as between the central and peripheral nervous system. Our mental representations, traditionally designated as "cognitive", clearly contain aspects of perception and action, most obviously present in the neural signatures of mental simulation, and likely in many other cognitive operations, especially in the case of music processing, which inherently engages multiple senses and bodily responses, from tears and shivers (cf. Sloboda, 1991) to induced movement (Janata et al., 2012). The predictive processing ideas, while possibly uniting apparent contradictions within the concepts of embodied cognitive science, concerning what the specific computational role of the body is in cognition (Kiverstein, 2012), implicate something like multimodal imagination to be at play continuously during music processing (and perception more generally), impacting the interpretation of perceptual signals as well as the performance of movements. This closely resembles how imagery has been considered from the perspective of music perception research as well as performance science. Imagery skills tend to be thought of as a mark of musicianship (and thus appear in several historical tests of musical aptitude, see for instance Gordon, 1965; Seashore, 1915), although this more often relates to skills in deliberate sensory imagery rather than interpretative perceptual skills. Of course the two are likely related (to each other, and to musical training, see Pecenka & Keller, 2009), but are arguably phenomenologically different processes. The constructive imagery that is part of tracking and predicting one's environment is dependent on successful model construction through experience, whereas deliberate imagery, while making use of models or representations, presumably also requires attention and skill. This argues for experimental investigations that isolate task or modality effects, i.e. effects related to moving, perceiving, or imagining in a specific modality, from stimulus effects, based on the content of the action, percept or image (see for instance Daselaar et al., 2010; Schaefer, Vlek & Desain, 2011b).

The use of deliberate imagery in music has been described in the context of a number of goals, mostly focusing on mental rehearsal, but also the use of metaphors in acquiring technique or attaining expressive performance, and even imagery related to attenuating performance anxiety and stress (Gregg, Clark & Hall, 2008). The fact that these goals are all related to motor output and include sensory representations further underscores the aforementioned point on the multimodal nature of these representations. These deliberate types of imagery, and specifically mental rehearsal, are likely very closely related to musical expertise, which is a reason for many studies of sensory imagery to use trained musicians as participants, and may also be true for different domains of expertise (see Schaefer, 2014b for more discussion). Mental rehearsal is reported as an effective aid in learning music, athletics, dance and used in movement rehabilitation and brain-computer interfaces (Gerven et al., 2009; Owen et al., 2006), but is also dependent on sensory imagery skills (Clark & Williamon, 2011), which has been identified as a problem for its application in rehabilitation (Malouin & Richards, 2010). Somewhat conflicting results can be found in the literature for other modalities of imagination, such as for instance athletes (Ozel, Larue, & Molinaro, 2004) and dancers (Malouin et al., 2009) reporting more vivid deliberate movement imagery than non-experts, while dancers counter-intuitively show slower performance in mental rotation (Jola & Mast, 2005), the latter finding interpreted as due to a difference in strategy. This interpretation is supported by findings illustrating that for some aspects of spatial abilities, such as mental rotation, a range of strategies may be used, that may also impact task ability (Logie, Pernet, Buonocore, & Della Sala, 2011). Again, the difference between active sensory or motor imagery and constructive processes such as mental rotations and manipulations appear to dissociate (as already reported by Farah, Weisberg, Monheit, & Peronnet, 1989), supporting the use of conceptual subtypes such as sensory and constructive imagination.

The role of mental representations in action-perception loops, such as the coordination of behavioral output to an external stimulus, is thus mainly one involving the generation and evaluation of predictions. The constructive imagery that is a part of active perception is still a somewhat elusive phenomenon that in some cases happens completely subconsciously, but in other cases emerges spontaneously into our awareness. Examples are the spontaneous but conscious imagery experiences involved in auditory anticipation (Leaver et al., 2009) or gap filling (Kraemer, Macrae, Green, & Kelley, 2005), indicating that the distinctions between imagery types are not always clear. Hargreaves (2012) makes a case for viewing imagery as a central aspect of music perception, formulating a three-way reciprocal feedback model of music processing in which the agent, be it a listener, performer, composer or improviser interacts with the musical material on one hand and the situation or context on the other (which are also bidirectionally connected), with imagination forming the core of the model, impacting both perception and production and being credited with something like musical executive function, crucial to musical creativity. For piano performance, deliberate music imagery has recently been described as a great aid in practice and memorization, but also offering performers a way to reduce cognitive load while performing, once a detailed representation of a piece, including auditory, visual and bodily aspects, is present (Davidson-Kelly, 2014).

While the terminology might still benefit from further specification, this way of representing the role of imagery in music processing aligns closely with the view described here, with an explicit focus on the personalized nature of musical experiences. Researching this, and other types of imagery, will likely offer more clarification of imagery subtypes, the extent to which the subtypes may overlap or co-occur, and the nature of the representations that facilitate difference types of imagery. Given the practical uses of mental practice, a better understanding of the skill set that is involved in this type of imagination would have direct practical implications. Further investigation of the individual differences in constructive imagery, potentially related to other musical sub-skills (cf. Pecenka & Keller, 2011), appears integral to a better understanding of musical experiences. Finally, given the great potential for music to facilitate shared experience (cf. Overy, 2012) the concept of shared mental models, be it through a commonality in previous exposure or the current listening situation, may offer a theoretical framework for potential investigations of rich interpersonal experiences in musical settings.

Conclusion

To summarize, the divergent use of the term 'imagery' in scientific literature groups tasks together that can not reasonably be considered to be functionally the same, as the level to which they are effortful or automatic, as well as the extent to which they are consciously experienced, vary greatly. By specifying our terms more clearly, more progress can be made towards an understanding of the mechanism of imagery phenomena, and their role in other cognitive, perceptual and bodily processes. Importantly, the representations that underlie imagery phenomena should be recognized to be multimodal, treating the brain and the body as one system. A theoretical account that could drive future research is that of predictive processing, which has already been established as highly suited to musical behavior. The concept that mental models underlie perception but also various types of imagery provides a potential explanation for the shared neural activation between deliberate as well as automatic imagery tasks with actual perception or action.