WITH "Shaping and Co-Shaping Forms of Vitality in Music: Beyond Cognitivist and Emotivist Approaches to Musical Expressiveness" Dr. Jin Hyun Kim proposes a conceptual framework for empirical research on aesthetics, in particular the aesthetics of music. The topic of the proposal accords with current discourse on aesthetics, emotions, consciousness, and empathy in philosophy, psychology, and the neurosciences (Bacci & Melcher, 2011; Coplan & Goldie, 2011; Schellekens & Goldie, 2011; Shimanura & Palmer, 2012). It is also related to current research on music, meaning, gesture, movement (Godøy & Leman, 2010; Gritten & King, 2011), and communicative musicality (Malloch & Trevarthen, 2009).

At present, meaningfulness of music is mainly discussed focusing on sound, emotions, and aesthetic emotions (Koelsch, 2013; Sander, 2013; Silvia, 2005, 2009; Robinson, 2009, 2010;). Kim's analysis of current discussions in research on musical emotions and aesthetics in philosophy, psychology, and cognitive neuroscience reveals a general empiricist stance associated with an epistemological subject-object dualism leading to a methodological activity-passivity dichotomy in empirical research, in particular in experimental design; in response to these epistemological and methodological problems, she offers a framework extending such conceptualizations of music and aesthetics. The proposal's core idea is that musical expressiveness and aesthetic experience are shaped or formed during a continuous process of realization, i.e. during actions of production and reception. Musical expressiveness, forms of vitality, and shaping/co-shaping are introduced as key concepts. "Musical expressiveness" indicates that the proposed approach to musical aesthetics encompasses appreciation as rooted in a subject, i.e. a person's phenomenal experience, and an object, i.e. musical sound. "Forms of vitality" provide a means to reconcile the epistemological subject and object of expressiveness, i.e. phenomenological experience and musical sounds. The meaningfulness of aesthetic experience is created—shaped and co-shaped—by humans in a process of continuous realization; the interactive processes that are the shaping and co-shaping of forms of vitality constitute the basis for (re)creating meaningful aesthetic experiences leading to musical expressiveness.

Notably, in general, this framework is in accordance with genetic epistemology (Piaget, 1970/1983) and with a schema-based action-oriented approach to action, music, and language (Arbib, 2003, 2013). Furthermore, Kim proposes developing an approach to experimental research in neurophenomenology that deals with aesthetic experience from a first-person perspective. The conception of aesthetic experience based on shaping and co-shaping is a promising basis for the development of a feasible empirical research strategy as well as ideas for corresponding experimental designs. In total, Kim's proposal entails research on interaction, processes, and phenomenological experience. To this end, a theory of processing and interaction in connection with a theory of consciousness is necessary, which, to my mind, presents three further requirements for empirical aesthetics: 1) Integration of computational cognitive modeling in addition to the development of experimental methods for studying mental processes (Bower & Clapper, 1989); 2) computational models of emotional processes related to music and aesthetics; and 3) a methodology for phenomenology in empirical research and experiments.


Efforts at analyzing and explaining interactions and processes, as well as testing proposed models and theories, require computational cognitive modeling (Sun, 2008). In terms of an action-oriented approach to music and aesthetics, it is necessary to give empirical evidence for a functional architecture consisting of schemas, and to strive for an explanation of their dynamics; those dynamics can be studied by investigating processes, which are realized by mechanisms carrying out specific operations. These mechanisms are biophysically observable phenomena implementing operations; operations—e.g., attention, empathy, emotion, memory—are hypothetical constructs, theoretical concepts to be related to biophysical processes or structures. Despite several theoretical problems concerning the explanation of the concept of computation, these operations—as parts of a functional architecture and potential processes in time—are currently best conceived of as (effective) procedures, i.e. mathematical and logical descriptions (of empirical phenomena) subsumable under the concept of Turing-machine computability.

My claim, then, is that computational cognitive modeling using robots is necessary to test the empirical adequacy of theories of processing and interaction. Computational cognitive modeling provides a feasibility test for hypotheses on the realizability of processes and interactions implied by a certain model. If computational modeling is used in connection with robots, the feasibility test becomes a test of empirical adequacy—real-time exhibition of a task in our world. Successfully dancing a sarabande, for example, could be a robot's task in testing the empirical adequacy of a dynamic systems approach to attention and rhythm perception in music (Large, 2010; Port, 2003) using computational cognitive modeling with a robot. In general, successfully testing a model in a scenario involving robots is evidence of its real-world realizability, and the model not only serves as a summary of empirical data with some predictive capacity that relies only on plausibility arguments for its empirical relevance but becomes an empirically confirmed hypothesis. Another more general advantage of computational cognitive modeling is that the metaphor "cognition as computational process or computation" often used in (cognitive) psychology and the neurosciences becomes a scientifically relevant concept providing theoretically and empirically convincing explanations. In addition, computational cognitive modeling using robots provides a bridge to frameworks for more rigorous analysis of theories and formal descriptions of processes and interactions (cf. Wells, 2006). These formal theories or concepts in turn might be used metaphorically as heuristics in empirical research and their empirical adequacy tested by computational cognitive modeling.

As long as such functional architectures have not been successfully tested under real-world conditions—specifically, implemented with robots or virtual agents interacting with our world—their empirical adequacy is left unsettled and open to questioning of its scientific validity. Computational cognitive modeling using robots is thus essential for testing the empirical adequacy of mathematical models, explicit theories, and formal models of processes and interaction; to my mind, it is at present the most appropriate method for testing the empirical adequacy of theories of processing and interaction, as well as crucial in the development of such theories in the first place.


In aesthetics and music—and cognition in general (Pessoa, 2009)—the role of emotions cannot be neglected. In developing a theory of aesthetic or music appreciation which takes the dynamics of processes and interactions into account, it is important to study emotions as modes of operations instead of entities (Fellous, 2006). The existing important models of emotional processing, Scherer (2010) and Sander (2013), provide hypotheses about functional architectures; however, the alleged interactions of their components and time courses are merely metaphorical descriptions of the processes involved. Their empirical adequacy as models or theories of processing thus needs to be tested in future research.

To get a clearer view of the dynamics of musical and aesthetic emotions, experimental research is necessary, but insufficient by itself. Testing current theories based on conceptual analysis and empirical evidence—in pursuit of developing ideas about the dynamics of aesthetic emotions as feelings, their role in social interaction, and their cultural embedding—requires computational modeling of emotional processes. Much work remains to be done towards this end, in general and in research on music in particular. In music research, computational cognitive modeling is still in its infancy; recent surveys also reveal that computational models of musical emotion processing are lacking (Eerola, 2012; Purwins, Grachten, Herrera, Hazan, Marxer, & Serra, 2008; Purwins, Herrera, Grachten, Hazan, Marxer, & Serra, 2008; Temperley, 2013). The complexity of the situation is further heightened because for emotional processing in aesthetics and music, the role of phenomenological consciousness, social interaction, and cultural situatedness needs to be taken into account. 1 A schema theoretic and action-oriented approach to empirical aesthetics and music needs to be extended to research on social schemas to enable this (Seifert, 2011).

Several first proposals for principles or guidelines concerning computational modeling of emotion, in particular emotion generation and effect, exist (Fellous & Ledoux, 2005; Gunes & Schuller, 2013; Hudlicka, 2011; Marsella, Gratch, & Petta, 2010; Roesch, Korsten, Fragopanagos, Taylor, Grandjean, & Sander, 2011). New insights into and stimulation of research on aesthetics and music are offered by action-oriented approaches and situated cognition (Robbins & Aydede, 2009; Seifert, 2011; Seifert et al., 2013), affective computing (Gökçay & Yildirim, 2011; Scherer, Bänziger, & Roesch, 2010), human-robot interaction (Dautenhahn, 2007), and musical robotics (Solis & Ng, 2011)—or more generally, research on social human-agent interaction, in particular in New Media Art and entertainment—in connection with cultural and social cognitive neuroscience (Han, Northoff, Vogeley, Wexler, Kitayama, & Varnum, 2013) as well as phenomenological approaches in experimental research (cf. Albertazzi, 2013).


Empirical research on aesthetics, music, and emotion cannot avoid taking consciousness and experience, i.e. a first-person perspective or the phenomenological mind (Jackendoff, 1987), into account (Clarke & Clarke, 2011; Leman, 2008; Northoff, 2012; Shimanura & Palmer, 2012; Tsuchiya & Adolphs, 2007). Although there is an enormous number of techniques available in the current methodology of psychology, an explicit methodology of experiential descriptions to study consciousness and phenomenal experience is lacking; in experimental and cognitive psychology and a fortiori in cognitive neuroscience there are almost no methods taking the first-person perspective into consideration (Gadenne, 1996, p. 13).

Kim is well aware of such methodological desiderata and proposes a neurophenomenological approach, which might offer the possibility of linking descriptions of phenomenological experiences with neuroscientific data of their neural substrates. To advance methodological thinking and develop new experimental designs in current music research and aesthetics, a discussion about introspection, phenomenological methods and practical applications in experimental contexts seems necessary. Some remarks on experimental phenomenology are here presented to point to possible topics for future discussion, and to add ideas about the role of phenomenology in cognitive science and empirical research (cf. Petitot, Varela, Pachoud, & Roy, 1999).

Rainer Mausfeld (2011; 2013) advocates experimental phenomenology in research on perception and draws attention to the importance of careful phenomenological descriptions in investigating fundamental principles of perception. Such descriptions should precede theoretical and experimental considerations, and "achievements must be not contaminated with physical or physiological considerations" (Mausfeld, 2013, p. 95). In accordance with Mausfeld and to ensure greater reliability of introspective reports in neuroscientific investigations, Shallice and Cooper (2011, p. 443) propose initially using phenomenological descriptions, followed by "correlat[ion] with cognitive neuroscience evidence." They conclude that the subject's expertise is relevant to the task, and that degree of expertise "should depend on the type of task." This raises methodological questions for experimental studies investigating aesthetic experience: What tasks might be relevant to investigate aesthetic experience, and thus should be used in experimental design? Depending on the task, what degree of expertise should the subjects have? How could that degree be determined? What are relevant experiences to focus on?

Experimental phenomenology requires methodological reconsiderations about experimental research (cf. Kuboy, 1999, p. 347). These reconsiderations might take phenomenological descriptions, such as those of the (philosophical and psychological) phenomenology in the Husserlian tradition, into account (Bischof, 2009; Dreyfus, 1982; Petitot et al., 1999). For example, such descriptions could be used to modify or refine experimental and modeling studies. Considerations in experimental phenomenology can be found in Gestalt or act psychology as developed by Carl Stumpf, Vittorio Benussi, Liliana Albertazzi, Giovanni Vicario, and Paolo Bozzi, to name just a few. In empirical research investigating mental processes, phenomenological reports might also be used to reconstruct the conceptual framework determining the (cultural and social) constraints involved in structuring aesthetic processing. Then, such reports might offer the possibility of discovering or at least giving some hints about relevant general structures, entities, or operations involved in the functional architecture underlying actual mental processing.


The previous remarks outline the challenging and highly complex problems in the study of empirical aesthetics focusing on processes and interactions, and the consequent urgent need to develop appropriate concepts, tools and empirical methods. New theoretical concepts and research methods for the acquisition of relevant data and tools for analysis are required to advance empirical aesthetics conceptually and empirically as the theory of processing and interaction. While advancing the field calls for an integration of experimental research and computational cognitive modeling, such modeling of emotional processes in aesthetics and music at present is lacking. Moreover, empirical aesthetics has to address methodological problems concerning research on the phenomenal mind, such as phenomenal consciousness, second-order consciousness, intentionality, and first-person perspective. As a result, computational modeling of emotional processes is particularly essential.

Computational cognitive modeling, situated cognition, affective computing, musical robotics, and human-robot interaction—in connection with experimental phenomenology, and social and cultural neuroscience—will provide new data and insights and thus will, in the near future, have a stimulating impact on empirical aesthetics, music research, and research on the (individual and collective) mind. A joint endeavor of theorists, experimentalists, and modelers from different disciplines will be necessary to deal with these challenges and possibilities.


  1. For a brief discussion about the possibility of feelings in animals and animats, i.e. machines, cf. Cruse (2009, pp. 185-190).
    Return to Text


  • Albertazzi, L. (Ed.) (2013). Handbook of Experimental Phenomenology: Visual Perception of Shape, Space, and Appearance. Chichester: Wiley.
  • Arbib, M.A. (2003). Schema theory. In: M.A. Arbib (Ed.), The Handbook of Brain Theory and Neural Networks. Cambridge, MA: MIT Press, pp. 993-998.
  • Arbib, M.A. (Ed.) (2013). Language, Music, and the Brain: A Mysterious Relationship. Cambridge, MA: MIT Press.
  • Bacci, F., & Melcher, D. (Eds.) (2011). Art and the Senses. Oxford: Oxford University Press.
  • Bischof, N. (2009). Psychologie: Ein Grundkurs für Anspruchsvolle. Stuttgart: Kohlhammer.
  • Bower, G.H., & Clapper, J.P. (1989). Experimental methods in cognitive science. In: M.I. Posner (Ed.), Foundations of Cognitive Science. Cambridge, MA: MIT Press, pp. 245-300.
  • Clarke, D., & Clarke, E. (Eds.) (2011). Music and consciousness: philosophical, psychological, and cultural perspectives. Oxford: Oxford University Press.
  • Coplan, A., & Goldie, P. (Eds.) (2011). Empathy: Philosophical and Psychological Perspectives. Oxford: Oxford University Press.
  • Cruse, H. (2009). Neural Networks as Cybernetic Systems. 3rd and Revised Edition. Bielefeld: Brain, Minds & Media.
  • Dautenhahn, K. (2007). Socially intelligent robots: dimensions of human-robot interaction. In: N. Emery, N. Clayton, & C. Frith (Eds.), Social Intelligence: From Brain to Culture. Oxford: Oxford University Press, pp. 313-351.
  • Dreyfus, H.L. (Ed.) (1982). Husserl, Intentionality, and Cognitive Science. Cambrigde, MA: MIT Press.
  • Eerola, T. (2012). Modeling listeners' emotional response to music. Topics in Cognitive Science, Vol. 4, No. 4, pp. 607-624.
  • Fellous, J.-M. (2006). A mechanistic view of the expression and experience of emotion in the arts (book review of Deeper Than Reason: Emotion and Its Role in Literature, Music, and Arts by Jenefer Robinson, Oxford: Oxford University Press, 2005). American Journal of Psychology, Vol. 119, No. 4, pp. 668-674.
  • Fellous, J.-M., & Ledoux, J.E. (2005). Toward basic principles for emotional processing: what the fearful brain tells the robot. In: J.-M. Fellous, & M.A. Arbib (Eds.), Who Needs Emotions? The Brain Meets the Robot. Oxford: Oxford University Press, pp. 79-115.
  • Gadenne, V. (1996). Bewußtsein, Kognition und Gehirn – Einführung in die Psychologie des Bewußtseins. Bern: Hans Huber.
  • Godøy, R.I., & Leman, M. (Eds.) (2010). Musical Gestures: Sound, Movement, and Meaning. New York: Routledge.
  • Gökçay, D., & Yildirim, G. (Eds.) (2011). Affective Computing and Interaction: Psychological, Cognitive and Neuroscientific Perspectives. Hershey: Information Science Reference.
  • Gritten, A., & King, E. (Eds.) (2011). New Perspectives on Music and Gesture. Farnham: Ashgate.
  • Gunes, H., & Schuller, B. (2013). Categorical and dimensional affect analysis in continuous input: Current trends and future directions. Image and Vision Computing, Vol. 31, No. 2, pp. 120-136.
  • Han, S., Northoff, G., Vogeley, K., Wexler, B.E., Kitayama, S., & Varnum, M.E.W. (2013). A cultural neuroscience approach to the biosocial nature of the human brain. Annual Review of Psychology, Vol. 64, pp. 335-359.
  • Hudlicka, E. (2011). Guidelines for designing computational models of emotion. International Journal of Synthetic Emotions, Vol. 2, No. 1, pp. 26-79.
  • Jackendoff, R. (1987). Consciousness and the Computational Mind. Cambridge, MA: MIT Press.
  • Koelsch, S. (2013). Music and emotion. In: J. Armony & P. Vuilleumier (Eds.), The Cambridge Handbook of Human Affective Neurocscience. Cambridge: Cambridge University Press, pp. 286-303.
  • Kubovy, M. (1999). Gestalt psychology. In: R.A. Wilson & F.C. Keil (Eds.), The MIT Encyclopedia of the Cognitive Sciences. Cambridge, MA: MIT Press, pp. 346-349.
  • Large, E.C. (2010). Neurodynamics of music. In: M.R. Jones, R.R. Fay, & A.N. Popper (Eds.), Music Perception. New York: Springer, pp. 201-232.
  • Leman, M. (2008). Embodied Music Cognition and Mediation Technology. Cambridge, MA: MIT Press.
  • Malloch, S., & Trevarthen, C. (Eds.) (2009). Communicative Musicality: Exploring the Basis of Human Companionship. Oxford: Oxford University Press.
  • Marsella, S., Gratch, J., & Petta, P. (2010). Computational models of emotion. In: K.R. Scherer, T. Bänziger, & E. Roesch (Eds.), A Blueprint for Affective Computing: A Sourcebook and Manual. Oxford: Oxford University Press, pp. 21-41.
  • Mausfeld, R. (2011). Intrinsic multiperspectivity: Conceptual forms and the functional architecture of the perceptual system. In: W. Welsch, W.J. Singer, & A. Wunder (Eds.), Interdisciplinary Anthropology: Continuing Evolution of Man. Berlin: Springer, pp. 19-54.
  • Mausfeld, R. (2013). The attribute of realness. In: L. Albertazzi (Ed.), Handbook of Experimental Phenomenology: Visual Perception of Shape, Space, and Appearance. Chichester: Wiley.
  • Northoff, G. (2012). From emotions to consciousness: a neuro-phenomenal and neuro-relational approach. Frontiers in Psychology: Emotion Science, Vol. 3, No. 303, pp. 1-17.
  • Pessoa, L. (2009). Cognition and emotion. Scholarpedia, Vol. 4, No. 1, p. 4567.
  • Petitot, J., Varela, F.J., Pachoud, B., & Roy, J.-M. (Eds.) (1999). Naturalizing Phenomenology: Issues in Contemporary Phenomenology and Cognitive Science. Stanford, CA: Stanford University Press.
  • Piaget, J. (1970/1983). Piaget's theory. In: P. Mussen (Ed.), Handbook of Child Psychology, Vol. I. 4th Edition. New York: Wiley.
  • Port, R.F. (2003). Dynamical systems hypothesis in cognitive science. In: L. Nadel (Ed.), Encyclopedia of Cognitive Science. Hoboken, NJ: Wiley, pp. 1-6.
  • Purwins, H., Grachten, M., Herrera, P., Hazan, A., Marxer, R., & Serra, X. (2008). Computational models of music perception and cognition II: Domain-specific music processing. Physics of Life Review, Vol. 5, No. 3, pp. 169-182.
  • Purwins, H., Herrera, P., Grachten, M., Hazan, A., Marxer, R., & Serra, X. (2008). Computational models of music perception and cognition I: The perceptual and cognitive processing chain. Physics of Life Review, Vol. 5, No. 3, pp. 151-168.
  • Robbins, P., & Aydede, M. (2009). A short primer on situated cognition. In: P. Robbins & M. Aydede (Eds.), The Cambridge Handbook of Situated Cognition. Cambridge: Cambridge University Press, pp. 3-10.
  • Robinson, J. (2009). Aesthetic emotions (philosophical perspectives). In: D. Sander & K.R. Scherer (Eds.), The Oxford Companion to Emotion and the Affective Sciences. Oxford: Oxford University Press, pp. 6-9.
  • Robinson, J. (2010). Emotional responses to music: What are they? How do they work? And are they relevant to aesthetic appreciation? In: P. Goldie (Ed.), The Oxford Handbook of Philosophy of Emotion. Oxford: Oxford University Press, pp. 651-680.
  • Roesch, E.B., Korsten, N., Fragopanagos, N.F., Taylor, J.G., Grandjean, D., & Sander, D. (2011). Biological and computational constraints to psychological modeling of emotion. In: P. Petta, C. Pelachaud, & R. Cowie (Eds.), Emotion-Oriented Systems: The Humaine Handbook. Berlin: Springer, pp. 47-62.
  • Sander, D. (2013). Models of emotion: the affective neuroscience approach. In: J. Armony & P. Vuilleumier (Eds.), The Cambridge Handbook of Human Affective Neuroscience. Cambridge: Cambridge University Press, pp. 5-54.
  • Schellekens, E., & Goldie, P. (Eds.) (2011). The Aesthetic Mind: Philosophy and Psychology. Oxford: Oxford University Press.
  • Scherer, K.R. (2010). The component process model: Architecture for a comprehensive computational model of emergent emotion. In: K.R. Scherer, T. Bänziger & E.B. Roesch (Eds.), Blueprint for affective computing: A sourcebook. Oxford: Oxford University Press, pp. 47-70.
  • Scherer, K.R., Bänziger, T., & Roesch, E.B. (Eds.) (2010). Blueprint for Affective Computing: A Sourcebook. Oxford: Oxford University Press.
  • Seifert, U. (2011). Investigating the musical mind: situated cognition, artistic human-robot interaction design, and cognitive musicology. Transhumanities, Vol. 4, No. 1, pp. 149-161.
  • Seifert, U., Verschure, P.F.M.J., Arbib, M.A., Cohen, A.J., Fogassi, L., Fritz, T., Kuperberg, G., Manzoli, J., & Rickard, N. (2013). Semantics of internal and external worlds. In: M.A. Arbib (Ed.), Language, Music and the Brain: A Mysterious Relationship. Cambridge, MA: MIT Press, pp. 203-229.
  • Shallice, T., & Cooper, R.P. (2011). The Organization of Mind. Oxford: Oxford University Press.
  • Shimanura, A.P., & Palmer, S.E. (Eds.) (2012). Aesthetic Science: Connecting Minds, Brains, and Experience. Oxford: Oxford University Press.
  • Silvia, P.J. (2005). Emotional responses to art: From collation and arousal to cognition and emotion. Review of General Psychology, Vol. 9, No. 4, pp. 342-357.
  • Silvia, P.J. (2009). Aesthetic emotions (psychological perspectives). In: D. Sander & K.R. Scherer (Eds.), The Oxford Companion to Emotion and the Affective Sciences. Oxford: Oxford University Press.
  • Solis, J., & Ng, K. (Eds.) (2011). Musical Robotics and Interactive Multimedia Systems. Berlin: Springer.
  • Sun, R. (Ed.) (2008). The Cambridge Handbook of Computational Psychology. Cambridge: Cambridge University Press.
  • Temperley, D. (2013). Computational models of music cognition. In: D. Deutsch (Ed.), The Psychology of Music. 3rd Edition. New York: Academic Press, pp. 327-368.
  • Tsuchiya, N., & Adolphs, R. (2007). Emotion and consciousness. Trends in Cognitive Sciences, Vol. 11, No. 4, pp. 158-167.
  • Wells, A.J. (2006). Rethinking Cognitive Computation: Turing and the Science of Mind. New York: Palgrave.
Return to Top of Page