RESEARCH on perceived emotion in music generally relies on listeners’ judgments of aesthetic qualities based on audio excerpts of varying lengths. [1] Such stimuli may involve live or pre-recorded human performances or computer-generated synthetic audio. Eerola and Vuoskoski (2013) estimate that a majority (75%) of studies in music and emotion research use human performances. Research on perceived emotion in music often investigates how specific parameters such as melody, harmony, and rhythm affect listeners’ judgments of aesthetic qualities. Researchers face a methodological problem of whether to use “real music” or “laboratory music” as stimuli for their investigations (Jones, 2010). We define “laboratory music” to be any music or sounds (e.g., pure sine tones) created to satisfy some researcher-imposed experimental and statistical constraints. “Real music” is, then, everything else, though in this paper we focus particularly on human performances of Western art music from 1893 to 1965. On the one hand, by using real music, researchers can ensure that the stimuli are ecologically valid and represent potential real-life musical experience—a quality that cannot be afforded by more controlled audio signals that consist of simple tone patterns. Ecological validity is also limited in stimuli that are generated via pitch randomization. On the other hand, one important disadvantage of using human performances of real music as compared with computer-generated synthetic audio—whether ordered or randomized—is the lack of experimental control on the stimuli, which may limit researchers’ ability to manipulate source materials and generalize findings. One solution to this problem may be to extract performance data related to the “expressive performance actions” of human performers, that is, micro-differences in terms of tempo, dynamics, articulation, intonation, and vibrato (Kirke & Miranda, 2009), and investigate their influence on listeners’ experience. However, composers often use several cues to create a specific effect. For example, softer dynamics, legato articulation, and slow tempo may be applied to a given pitch structure to suggest nostalgia, while louder dynamics, staccato articulations, and a faster tempo may be combined to convey urgency. Using laboratory music allows researchers to study the specific impact of a set of musical parameters by controlling for other factors (e.g., Scherer & Oshinsky, 1977). By testing several cues independently as well as in combination, researchers have been able to rank the effect of each type of cue and characterize their effect as linear and additive (Eerola et al., 2013; Juslin & Linström, 2010). Nonetheless, while production mode (e.g., human, machine, or computer) and source materials (e.g., real music, melodies composed by the reported along with the results, it is not clear whether the methods used to produce musical stimuli have an effect on listeners’ experience.

INTRODUCTION

In this study, we offer a new approach that balances the need for ecological validity with that of controlled stimuli. Using excerpts of pre-composed music, we create partially controlled testing conditions and subsequently explore the influence of uncontrolled musical features on participants’ emotional ratings. In Experiment 1, participants rated perceived emotions in naturalistic pre-recorded human performances and computer-generated synthetic rendering of the same musical excerpts, while in Experiment 2 participants rated perceived emotions in computer-generated synthetic rendering of musical excerpts with ordered and randomized pitch presentation. Each experiment was then followed with a post-hoc, exploratory regression analysis of nine musical features (described below).

Perceived Emotion in Human Performance versus Synthetic Audio

Research related to audio generation in terms of performance medium tends to focus on two aspects: timbral differences and expressive differences. Studies on timbral differences typically ask participants to identify and categorize single-note stimuli in terms of instrument type as well as perceptual dimensions such as “nasality,” “brilliance,” and “naturalness” (Kendall et al., 1999). Other studies investigate the effect of timbre on emotion by comparing excerpts played on different instruments (e.g., electronic synthesizer vs. human performances on piano, violin, and trumpet), with observed effects on listeners’ perceived emotions interpreted as being related to acoustical factors (Hailstone et al., 2009; Le Groux & Verschure, 2010), rather than performance medium.

Studies on expressive differences investigate expressive performance actions applied in human performances of pre-composed music, focusing on observed differences between the notated score and a human performance (Oore et al., 2020), or between different human performances of the same notated score (Repp, 1998). Some studies have explored the effects such differences have on listeners’ experience by manipulating human performances. For example, listeners have been shown to be able to distinguish between original and tempo-transformed versions of the same human performance (Honing, 2007). Synthetic stimuli with different levels of timing manipulations have also been used to explore listeners’ perceived “expressiveness” and “liveliness” (Hähnel & Berndt, 2010). The addition of other human-like expressive performance actions to synthetic audio, such as expressive dynamics, has been shown to result in higher ratings of “likeability” and “emotional expressiveness” (Kamenetsy et al., 1997).

However, very few studies have explored listeners’ ratings of emotional expression in synthetic audio as compared to human performances. [2] On the one hand, listeners have been found to have a negative bias in their ratings of expressive qualities of human performances presented as synthetic (i.e., “pseudo-synthetic” performances; see Hähnel & Berndt, 2010). On the other hand, synthetic generation of short melodies with human-like expressive differences in tempo, sound level, frequency spectrum, articulation, attack, vibrato, and timing has been shown to convey discrete emotions such as happiness, sadness, anger, fear, and tenderness as effectively as human performances of the same melodies (Juslin, 1997). Nevertheless, direct comparisons of listeners’ perceived emotion in human and synthetic performances of more complex musical structures, such as those afforded by multi-part compositions, are still needed.

Pitch Structure

In this series of experiments, we investigate how musical structure affects the way listeners perceive the emotions expressed by the music. Although scholarship on the relationship between musical structure and expression dates back to antiquity, modern scholarship on the topic has flourished since the 1990s. By applying appropriate statistical techniques to experimental data, researchers have been able to systematically examine the relationship between certain aspects of musical structure and perceived emotion.

Previous studies on the influence of pitch on perceived emotion have focused on pitch height, range, and contour. Some of these studies used pre-composed or naturalistic melodies (Kendall et al., 1999), while others used computer-generated melodic materials that follow a set of carefully determined criteria pertaining to start/end pitches, rhythmic structure, and key (Kawakami et al., 2013). These studies have identified several ways pitch structure is related to perceived emotion in music. For example, researchers have found that melodic contour—that is, whether pitches in a melody tend to go upward or downward—has an effect on perceived valence, with upward melodies positively correlated with valence (Gomez & Danuser, 2007). It has also been reported that an ascending melody is associated with the categorical emotion of “happiness” and a descending melody with “sadness” (Juslin, 2019). Studies have also found an effect of pitch height on perceived emotion. Melodies in a higher pitch register are associated with tenderness, nostalgia, and calmness, whereas those in a lower pitch register are associated with power and tension (Aljanaki et al., 2014). However, researchers have also found that perceived emotional characteristics of instrumental sounds vary nonlinearly with their pitch height when examining single-note sounds played on string instruments (Gilburt et al., 2016). Previous studies have also reported that dissonance level is positively correlated with arousal (Dellacherie et al., 2010; Gomez & Danuser, 2007) and negatively correlated with valence (Costa et al., 2004; ​​Dellacherie et al., 2010; Fritz et al., 2013; Kawakami et al., 2013).

Despite these findings, the effect of pitch structure more widely conceived is still lacking. Most studies focus on single-change events (i.e., how a single pitch change results in contour changes) or the substitution of an expected chord (e.g., tonic chord) for an unexpected one (e.g., Steinbeis et al., 2005). In multi-part music, how pleasing two or more tones sound together (i.e., harmonic dissonance) also needs to be considered. However, apart from studies that compare the effects of specific pitch collections such as major vs. minor and tonal vs. atonal on listeners’ emotional judgments (Daynes, 2011; Gagnon & Peretz, 2003), the effect of the global structure of pitches in multi-part musical excerpts has yet to be examined.

While randomization of pitches is used, few studies directly compare the effect of ordered and randomized pitch structure on listeners’ experience. Furthermore, studies that do make use of the two types of pitch structure tend to focus not on listeners’ emotional experience, but on other aspects of music perception. For example, Peretz et al. (2009) used familiar melodies and variants with semi-randomized pitch to locate neural correlates of familiar melodies. Prince et al. (2009) examined how an atonal context comprising a series of randomized pitches could affect listeners’ judgments on pitch height. They reported that a randomized pitch context causes listeners’ pitch judgments to be more sensitive to deviations from temporal regularity. Similarly, Sears et al. (2023) found that a “temporally incoherent” harmonic context created by scrambling the sequence of chords in an excerpt slows down listeners’ judgment on the intonation of a target chord.

In Experiment 2, we focus on the interaction between pitch structure and perceived emotion. We compare listeners’ ratings of five emotional dimensions (mood, energy, movement, dissonance, and tension) while listening to ordered versus pitch-randomized variants that used the same set of pitches as the source stimuli. We did this with the aim of opening a new avenue of research using musical excerpts that feature a greater variety of changes in pitch structure. The five emotional dimensions were chosen based on those most often used in studies on music and emotion. Mood and energy are the core affects that provide the basis of the two-dimensional model (Russell, 2003), a framework that accommodates many emotion words such as joy, sadness, anger, and fear. Movement, tension, and dissonance were selected based on the rhythmic and harmonic characteristics of the selected musical excerpts, which feature rhythmically complex and relatively dissonant pitch structures.

Rhythm in Multi-Part Textures

The materials we selected for this study are characterized by concurrent rhythmic patterns that cannot readily be mapped onto a single metric grid (i.e., “polyrhythm”). These materials were selected because they provide a naturalistic and rich environment within which listeners’ perceived emotions can be measured. To date, very little research has looked into how rhythmically complex music is aesthetically evaluated by listeners. When real music is used (as opposed to controlled “lab” stimuli), attention is devoted to general aspects of the musical compositions, such as tempo, loudness, timbre, and mode. Reports on finer-grain structural elements characterizing source materials or experimental stimuli are limited and the effects of these elements are generally not investigated.

Research on musical emotion has shown that tempo and event density are directly related to high arousal and perceived emotion (Balkwill & Thompson, 1999; Gabrielson & Juslin, 1996; Hevner, 1937). Shorter note values and faster tempi have also been shown to correlate with feelings of happiness, surprise, tension, and amusement (Fernández-Sotos et al., 2016). Enjoyment has been found to be correlated with tempo and rhythmic entrainment (Labbé & Grandjean, 2014), with regularity being more “pleasing” to the listener (Scherer & Zentner, 2001). However, beyond providing tempo information in terms of beats per minute, most studies do not offer enough details on the rhythmic structure of the selected music to allow for generalization over a wider range of music. Moreover, with the exception of studies on polyrhythm (e.g., Handel & Lawson, 1983; Handel & Oshinski, 1981), which typically consist of superposed pulse trains with limited pitch content, and studies on groove, which use a range of musical stimuli (synthetic or human) extracted from a limited range of musical styles with varying degrees of syncopation (Witek et al., 2014), most studies on the effect of rhythmic structure on listeners’ perceived emotions make use of single-part musical sequences.

EXPERIMENT I: PRODUCTION MODE

The primary goal of the first experiment was to determine whether production mode (human vs. synthetic) had an effect on listeners’ judgment of five emotional dimensions (mood, energy, movement, dissonance, and tension) for two different instrumentations (piano vs. string quartet). Because of the lack of previous studies that test the influence of production mode on listeners’ perceived emotion, we assumed a null hypothesis for human vs. synthetic stimuli. In contrast, based on previous findings (Hailstone et al., 2009; Le Groux & Verschure, 2010), we predicted a main effect of instrumentation.

The secondary goal was to explore the possible effects of a subset of musical features on participants’ ratings, including uncontrolled rhythmic features. To that end, we used a set of computational measures of rhythmic structure (audio file duration, composite normalized pairwise variability index [nPVI], composite event density, event density ratio, nPVI group difference, and nested ratio) as well as pitch organization (pitch range, pitch mean, and sonority dissonance). These measures are discussed in more detail in the “Measures” section. As our secondary goal was exploratory, we did not formulate any specific hypotheses. Informally, we expected event density to be positively correlated with mood, energy, and tension (Fernández-Sotos et al., 2016; Gabrielson & Juslin, 1996; Labbé & Grandjean, 2014), pitch range to be positively correlated with mood and energy, and pitch mean to be negatively correlated with energy (Gomez & Danuser, 2007). We also expected sonority dissonance to be predictive of perceived dissonance and tension (Krumhansl, 2002; Lehne, 2014).

Methods

PARTICIPANTS

Participants were recruited using an online survey implemented in Qualtrics. The survey was approved by the Ethics Review Board of the University of British Columbia (UBC), and shared through social media postings, email notifications to institutional and professional listservs, and the UBC Psychology Sona platform. One hundred and sixty-two participants with self-reported normal hearing completed the study; two participants were excluded from analysis due to those participants’ reported difficulty with English in everyday life. The results we report here are based on the responses of the remaining 160 participants. Gender distribution was uneven, with a large proportion of participants self-identifying as women (76%) as compared to men (21%); two participants self-identified as non-binary persons and three participants selected “prefer not to answer.” Participants’ age ranged from 18 to 59 years old (M = 23.28; SD = 7.29) and self-reported years of formal musical training ranged from 0 to 20 years (M = 5.89; SD = 5.09). A greater proportion of participants reported familiarity with the musical style represented by the excerpts (43%, as compared with 23% and 34% for no familiarity and “not sure” respectively), but much fewer participants reported familiarity through listening or performance of a specific excerpt (23%, as compared with 61% and 17% for no familiarity and “not sure” respectively). Finally, most participants listened to the excerpts using their computer’s built-in speakers (41%), followed by standard and noise-canceling headphones (26%) or earbuds (25%); a small proportion of participants reported using external speakers (8%), while only one participant reported using a phone speaker (0%).

MATERIALS

Sixteen musical excerpts from 12 different composers were selected from the Suter (1980) Corpus (Poudrier & Shanahan, 2018), ranging from 1893 to 1965 in terms of composition year (see Table 1). [3] Based on the availability of realistic audio synthesis and for contrast in timbre, we selected an equal number of short piano and string quartet examples. Because duration of audio stimuli has been shown to have an effect on listeners’ emotion judgments (Bachorik et al., 2009), excerpts were trimmed to a duration of 5 to 9 s (M = 7.16; SD = 1.11).

Table 1. Source musical compositions for experimental stimuli listed alphabetically by composer’s last name. There are eight excerpts for each instrumentation type.

Two audio versions of each example were prepared by the same highly-trained composer: a human performance extracted from a commercial recording randomly selected from available recordings in the Naxos Music Library, and a high-quality musical instrument digital interface (MIDI) rendering using the EastWest sound library. Audio files extracted from recorded examples were trimmed using Audacity to allow excerpts’ duration to be more precisely measured. The tempo of the synthetic examples was set to match the notated beat rate; in three cases where there was no notated tempo (R686x, R582, and R574), the tempo was calculated based on the human recorded performance of the same excerpt. A 0.2 s fade-out was applied to the end of each example to reduce the abruptness of the ending, and both audio file versions were then amplified or attenuated to a peak volume of -1 dB. All synthetic renderings were based on the musical excerpts encoded using kern. The tempo of synthetic examples was set to match the average tempo without rubato or expressive microtiming (i.e., the timing of individual note onsets or releases). Piano excerpts were rendered with the Steinway D virtual instrument, and three virtual instruments from the Symphonic Orchestra Gold were used for the string quartet excerpts (solo violin, solo viola, and solo cello, with corresponding individual samples for specific articulations, when needed). Fine-tuning of the synthetic audio involved subjective decisions, with the goal to match the corresponding human recordings as closely as possible. To match the acoustics of the human recordings, reverb was added to the piano examples using Logic’s Space Designer; it was not deemed necessary to add reverb to the string quartet examples, which fairly closely matched the acoustics of the human performances. The precise timing of raising and lowering the sustain pedal in the piano excerpts was also fine-tuned. For the string quartet excerpts, each instrument may also have a variety of similar articulation techniques available, so choosing between legato, legato expressive, and slur, or between staccato, short staccato, and spiccato had to be done subjectively. Because the same source materials were used for both of the synthetic and human versions, the specific musical parameters were the same across these conditions.

PROCEDURE

The experiment was conducted online using Qualtrics, with participants instructed to complete the tasks in one sitting, focusing only on doing the experiment, and in a quiet location or wearing noise-canceling earphones. Participants were instructed to listen to the excerpt in its entirety at least once, and then rate the excerpt using five seven-point Likert scales. Participants rated the perceived mood (negative–positive), energy (low–high), movement (very little–very much), dissonance (low–high), and tension (low–high), with “movement” referring to how much the participant felt that they could move along to the music.

First, participants provided consent and read the survey instructions. Participants completed a pre-experiment questionnaire on which they reported their age, gender, formal musical training, and English-language fluency. Prior to listening to the experimental stimuli, participants heard a short audio file during which they were instructed to adjust their volume to a comfortable level, and then completed two practice trials (one of each instrumentation type); these trials were based on different source materials and were not included in the analysis. At the end of the survey, participants were asked to report what listening device was used to complete the survey, and whether they were familiar with the musical style of the excerpts or with the excerpts themselves through listening or performance. Lastly, participants were given the opportunity to provide feedback and read a debriefing document.

DESIGN

Excerpt, instrumentation, and performance were treated as within-subjects variables. For instrumentation, each participant heard one version of each of the sixteen excerpts (piano and string quartet). For performance, the order of the experimental trials was randomized across musical excerpts so that each participant heard only one performance version (human or synthetic) of each of the sixteen. Instrumentation version (piano or string quartet) was pre-determined as excerpts were selected from the corpus according to their instrumentation. No attempt was made to re-compose a given stimulus in its opposite instrumentation version. Observations of excerpts were therefore independent. To avoid bias, participants were not informed of the type of performance they would hear.

MEASURES

In addition to participants’ ratings of the five dependent variables using seven-point Likert scales, we selected several measures derived from rhythmic and pitch structures to explore the relationship between musical features and participants’ perceived emotions. Rhythmic features required visualization and analysis of each excerpt, and assessment of the differences between concurrent rhythmic patterns. Instrumental parts were divided into two contrasting rhythmic groups (A and B) based on rhythmic similarity within the group and dissimilarity across groups, with the lowest part on the score assigned to Group A by default. The experimental excerpts include up to four instrumental parts; note that although piano excerpts are notated on two staves, each staff could include more than one part. Because the rhythmic design of a given instrumental part may vary over time, group attribution was performed at the measure level. To allow for comparison across musical excerpts, we used “composite” rhythms, i.e., the sequential presentation of event onsets, where an “event” includes all the notes that have the same onset time. Figure 1 illustrates the visual analytic markup for a sample used in the experiment. Group A notes are colored in red, while Group B notes are in blue. The top two staves are the original score and underneath are the extracted composite rhythms and number of events for Group A only, Group B only, Groups A and B combined (“composite”), and the intersection of Groups A and B (“coincidence”). The analytic markup shown in the musical example is automatically generated by the composite filter in Verovio Humdrum Viewer (Poudrier & Sapp, 2022).[4]

Table 2. Extracted rhythm and pitch measures with calculation method.

Table 2 presents the six rhythm features and three pitch features used to characterize musical structure. While composite event density provides a measure for the rate of presentation of events in the full texture, event density ratio measures the potential for metric conflict or ambiguity between rhythmic groups. The nPVI, a measure of the average durational variation between successive pairs of events (Patel & Daniele, 2003), is the basis for composite nPVI and nPVI group difference, which measure rhythmic regularity and the differential in regularity between groups, respectively. Finally, nested ratio is calculated based on the degree of coincidence between parts, and as such it provides a measure of the potential for an integrated percept.

Pitch register was assessed as both pitch mean (average pitch) and pitch range (interval between lowest and highest pitch). To measure sonority dissonance, each sonority was assigned a score (s_n) based on its most dissonant interval (Poudrier et al., 2024): octave/unison = 0, P5/P4 = 1, m3/M3/m6/M6 = 2, M2/m7 = 3, A4/d5/M7 = 4, m2 = 5. [6] Generally, consonances and weak dissonances (0–3) greater than an octave are given the same score as their simple equivalent; strong dissonances (4–5) are assigned a lower score (-1) above two octaves. The average sonority dissonance score was weighted by the duration (d_n) of the sonority. Table 3 provides a summary of the means and standard deviations for piano and string quartet excerpts. Because the same source materials were used for both the synthetic and human versions, all measures have the same values across synthetic and human conditions. Although piano and string quartet examples were different in terms of musical materials, they belong to the same historical period. The relative stylistic homogeneity of these musical excerpts was supported by a series of t tests. There was no statistically significant difference between piano and string quartet for each of the nine musical features (Table 3).

Table 3. Summary of means and standard deviations for experimental stimuli (Piano, n = 8; String Quartet,n = 8). Means are given on top with standard deviations in parentheses below.

ANALYSIS

We conducted statistical analysis in RStudio, with R version 4.1.1 and used the rstatix package for summary statistics, the broom package for summarizing linear models, and the car package for calculating Variance Inflation Factor (VIF) for linear models (R Core Team, 2021; Fox & Weisberg, 2019; Kassambara, 2022; Robinson et al., 2022). The tidyverse package was used for data formatting and processing (Wickham et al., 2019).

To address our primary goal of the effect of performance and instrumentation on participant ratings, we conducted a two-way multivariate analysis of variance (MANOVA) on the combined five dependent variables with performance and instrumentation as the independent variables. Point biserial correlations were used to explore the linearity between the five dependent variables and the two independent variables of performance and instrumentation.

To address our secondary goal, we performed exploratory multiple regression analyses. We opted for five regressions, one for each rated emotional dimension, to evaluate the effects of the musical features on each of those five emotions separately. A linear model was constructed between each dependent variable and the selected factors. Because participants’ ratings were done on a seven-point Likert scale, which can present problems of linearity, dependent variables were log-transformed using log(1+x). We considered three potential models: a model with all pitch and rhythm factors, a model with only rhythm factors, and a model with only pitch factors. We further compared the VIF of each model (see Table 4). We included VIF in our consideration because we anticipated our ecological stimuli to exhibit high covariance between musical features (Gomez & Danuser, 2007). Note that the musical features are measured on different scales as described in Table 2. In reporting the multiple regression analyses, we provide both the parameter estimate (unstandardized) and the standardized coefficients for the purpose of comparison of effect size.

RESULTS I

Participants rated 16 excerpts on five seven-point Likert scales (N = 2,560). The average rating for energy was the highest (M = 4.79; SD = 1.51), while those for dissonance (M = 3.50; SD = 1.68) and movement (M = 3.71; SD = 1.80) were the lowest. The average ratings for mood and tension were in the 4–5 range (M = 4.28 and 4.03; SD = 1.54 and 1.73 respectively).

Production Mode and Instrumentation

The main effect of performance was statistically significant, but small, F(1, 2556) = 2.89, p = .013, while the main effect of instrumentation was statistically significant and large, F(1, 2556) = 15.97, p < .001. There was also a significant, although relatively small, interaction between performance and instrumentation, F(1, 2556) = 3.32, p = .005.

Point biserial correlations were calculated between each dependent variable and performance (Human = 1; Synthetic = 2) as well as instrumentation (Piano = 1; String Quartet = 2). Performance was positively correlated with mood, r_pb(2558) = .05, p = .01, but negatively correlated with tension, r_pb(2558) = -.06, p = .003. Instrumentation was negatively correlated with mood, r_pb(2558) = -.09, p < .0001, but positively correlated with energy, r_pb(2558) = .11, p < .0001, and tension, r_pb(2558) = .13, p < .0001.

Effects of Musical Features

For Akaike Information Criterion (AIC), Bayesian Information Criterion (BIC), and VIF, we used the heuristic that lower values are better. As shown in Table 4, the model with all rhythm and pitch factors provided a better fit for each of the five dependent variables, while the VIF values associated with the selected factors (Table 5) were all below 4, which was deemed an acceptable degree of covariance.

All selected musical features were predictive of participants’ ratings for one or more emotional dimensions, with the best model accounting for 41% of the variance in participants’ ratings of energy (adjusted R² = .413), followed by tension (adjusted R² = .232), mood (adjusted R² = .211), dissonance (adjusted R² = .104), and movement (adjusted R² = .084). Parameter estimates and standardized coefficients for each of the five emotional dimensions are presented in Table 6. Standardized coefficients are used below to interpret the relative strength of the factors on participants’ ratings.

Table 4. Comparison of model fits using Akaike Information Criterion (AIC) and Bayesian Information Criterion (BIC) with all nine rhythm and pitch factors, a subset with only the six rhythm factors, and a subset with only the three pitch factors. The model with all rhythm and pitch factors provides a better fit, with 8% (movement) to 41% (energy) of the variance accounted for.

Emotion	Model	df	AIC	BIC	Adjusted R2	p value
Mood	All	2550	849	913	.211	< .001
	Rhythm	2553	1129	1175	.118	< .001
	Pitch	2556	1239	1268	.079	< .001
Energy	All	2550	-131	-66.55	.413	< .001
	Rhythm	2553	81.33	128	.362	< .001
	Pitch	2556	574	604	.226	< .001
Movement	All	2550	2712	2776	.084	< .001
	Rhythm	2553	2807	2853	.048	< .001
	Pitch	2556	2886	2915	.018	< .001
Dissonance	All	2550	2369	2433	.104	< .001
	Rhythm	2553	2512	2559	.051	< .001
	Pitch	2556	2516	2545	.049	< .001
Tension	All	2550	1858	1923	.232	< .001
	Rhythm	2553	2203	2250	.120	< .001
	Pitch	2556	2191	2220	.123	< .001

Table 5. Variance Inflation Factor (VIF) values for the nine rhythm and pitch factors selected. All values are below 4.

Factor	VIF
Audio file duration	2.73
Composite event density	2.34
Composite nPVI	1.41
Event density ratio	2.51
nPVI group difference	3.38
Nested ratio	2.12
Pitch mean	3.39
Pitch range	1.64
Sonority dissonance	2.54

Six of the nine factors were significant predictors of participants’ ratings of perceived mood. Except for event density ratio, these were negatively correlated, with nPVI group difference and sonority dissonance being the most reliable predictors of mood. Audio file duration, nested ratio, and surprisingly, pitch range, were not significant predictors. On the other hand, all but event density ratio, were significant predictors of participants’ perceived energy. Composite event density, pitch range, and sonority dissonance were positively correlated with energy, while composite nPVI, nPVI group difference, audio file duration, pitch mean, and nested ratio were negatively correlated. Finally, although the selected factors accounted for approximately only 8% of the variance in participants ratings of movement, seven out of the nine factors were predictive of a lower perceived impulse to move along with the music, with nPVI group difference, sonority dissonance, pitch mean, and composite nPVI being the most reliable predictors . Event density ratio, audio file duration, and pitch range were positively correlated with movement ratings.

Perceived dissonance and tension showed similar results, with a few differences. All but composite event density were significantly related to perceived dissonance. As expected, higher sonority dissonance was the strongest predictor of higher ratings in perceived dissonance, with nPVI group difference, pitch mean, and to a lesser extent, pitch range also being positively correlated. Nested ratio, event density ratio, composite nPVI, and audio file duration were negatively correlated with dissonance. Similarly, all but audio file duration were reliable predictors of perceived tension, with sonority dissonance being the strongest predictor. Other predictors of perceived tension were nPVI group difference, pitch mean, composite event density, and pitch range. Finally event density ratio, composite nPVI, and nested ratio were negatively correlated with perceived tension, with event density ratio showing the largest effect among these.

Table 6. Parameter estimates and standardized coefficients for rhythmic and pitch features for participants’ ratings of mood, energy, movement, dissonance, and tension.

Emotional Dimension	Parameter	Estimate	St. coefficient	St. Error	t value	Pr(>|t|)
Mood (adjusted R2 = .211)	(Intercept)	2.98	n/a	0.09	34.00	< .001
	Audio file duration	0.01	0.04	0.01	1.34	0.18
	Composite event density	-0.01	-0.05	0.004	-2.02	0.04
	Composite nPVI	-0.001	-0.04	0.0004	-2.04	0.04
	Event density ratio	0.41	0.22	0.05	8.01	< .001
	nPVI group difference	-0.01	-0.44	0.001	-13.77	< .001
	Nested ratio	0.01	0.01	0.05	0.30	0.76
	Pitch mean	-0.01	-0.31	0.001	-9.70	< .001
	Pitch range	-0.001	-0.03	0.001	-1.46	0.14
	Sonority dissonance	-0.18	-0.43	0.01	-15.36	< .001
Energy	(Intercept)	1.77	n/a	0.07	24.49	< .001
(adjusted R2 = .413)	Audio file duration	-0.03	-0.11	0.01	-4.52	< .001
	Composite event density	0.04	0.30	0.003	12.84	< .001
	Composite nPVI	-0.01	-0.29	0.0003	-16.37	< .001
	Event density ratio	-0.004	-0.002	0.04	-0.09	0.93
	nPVI group difference	-0.002	-0.17	0.0004	-5.98	< .001
	Nested ratio	-0.08	-0.05	0.04	-2.06	0.04
	Pitch mean	-0.003	-0.07	0.001	-2.46	0.01
	Pitch range	0.01	0.21	0.001	11.08	< .001
	Sonority dissonance	0.05	0.12	0.01	5.00	< .001
Movement (adjusted R2 = .084)	(Intercept)	2.35	n/a	0.13	18.66	< .001
	Audio file duration	0.05	0.11	0.01	3.61	< .001
	Composite event density	0.003	0.02	0.01	0.52	0.60
	Composite nPVI	-0.003	-0.10	0.001	-4.57	< .001
	Event density ratio	0.49	0.20	0.07	6.71	< .001
	nPVI group difference	-0.01	-0.37	0.001	-10.55	< .001
	Nested ratio	0.11	0.04	0.07	1.53	0.13
	Pitch mean	-0.01	-0.23	0.002	-6.69	< .001
	Pitch range	0.002	0.05	0.001	2.05	0.04
	Sonority dissonance	-0.16	-0.28	0.02	-9.28	< .001
Dissonance (adjusted R2 = .104)	(Intercept)	0.34	n/a	0.12	2.86	0.004
	Audio file duration	-0.03	-0.07	0.01	-2.29	0.02
	Composite event density	0.01	0.03	0.01	1.01	0.31
	Composite nPVI	-0.003	-0.11	0.001	-5.07	< .001
	Event density ratio	-0.24	-0.11	0.07	-3.55	< .001
	nPVI group difference	0.005	0.26	0.001	7.61	< .001
	Nested ratio	-0.38	-0.16	0.07	-5.71	< .001
	Pitch mean	0.01	0.26	0.002	7.53	< .001
	Pitch range	0.002	0.05	0.001	2.15	0.03
	Sonority dissonance	0.16	0.30	0.02	10.23	< .001
Tension (adjusted R2 = .232)	(Intercept)	0.35	n/a	0.11	3.24	0.001
	Audio file duration	-0.02	-0.05	0.01	-1.86	0.06
	Composite event density	0.03	0.17	0.01	6.45	< .001
	Composite nPVI	-0.003	-0.13	0.001	-6.20	< .001
	Event density ratio	-0.45	-0.20	0.06	-7.23	< .001
	nPVI group difference	0.005	0.27	0.001	8.61	< .001
	Nested ratio	-0.31	-0.13	0.06	-5.18	< .001
	Pitch mean	0.01	0.19	0.002	5.93	< .001
	Pitch range	0.003	0.08	0.001	3.69	< .001
	Sonority dissonance	0.24	0.47	0.01	16.89	< .001

DISCUSSION I

The primary goal of this study was to investigate more directly the influence of synthetic computer-generated audio as compared to pre-recorded human performances on listeners’ perceived emotions, within the context of rhythmically complex music excerpts that contrasted in timbre (piano and string quartet). Production mode was found to have a small but significant effect, with synthetic performances being perceived as more positive in mood and conveying less tension. This finding extends previous research which shows that synthetic generation of short melodies can convey discrete emotions effectively (Juslin, 1997), and further qualifies the effect of synthetic audio on listeners’ perceived emotions. One potential factor that might have a mitigating effect on the perception of tension is the absence of expressive variations in the timing of events in our synthetic excerpts as microtiming has been shown to alter listeners’ musical experience (Bhatara et al., 2011; Senn et al., 2016). As expected, instrumentation also had a significant and large effect, with string quartet excerpts giving rise to a more negative mood, greater energy, and higher perceived tension than piano excerpts. The main effect of instrumentation is consistent with research on the influence of timbre and acoustical factors on listeners’ perceived emotions (Farbood & Price, 2014; Hailstone et al., 2009). Because each participant heard each musical excerpt only in the piano or string quartet condition, and because these musical excerpts differed in terms of specific musical features, the presence of some hidden factor related to the instrumentation or musical excerpts’ features cannot be fully discounted and should be taken into consideration in future experiments (i.e., using musical examples that afford piano and string quartet performances). Also, it is not clear whether the quality of the vibrato produced by string instruments, which is difficult to synthesize, might have been a factor on participants’ ratings of the five emotional dimensions.

Table 7. Summary of standardized coefficients for nine rhythm and pitch factors for each of the rated emotional dimensions in Experiment 1, with significance levels for each. Significance levels are: * p < .05,

** p < .01, *** p < .001. Composite nPVI, nPVI group difference, pitch mean, and sonority dissonance are significant for all five emotional dimensions, with sonority dissonance and nPVI group difference being the strongest predictors.

Parameter	Standardized Coefficients
	Mood	Energy	Movement	Dissonance	Tension
Audio file duration	0.04	-0.11***	0.11***	-0.07*	-0.05
Composite event density	-0.05*	0.30***	0.02	0.03	0.17***
Composite nPVI	-0.04*	-0.29***	-0.10***	-0.11***	-0.13***
Event density ratio	0.22***	-0.002	0.20***	-0.11***	-0.20***
nPVI group difference	-0.44***	-0.17***	-0.37***	0.26***	0.27***
Nested ratio	0.01	-0.05*	0.04	-0.16***	-0.13***
Pitch mean	-0.31***	-0.07*	-0.23***	0.26***	0.19***
Pitch range	-0.03	0.21***	0.05*	0.05*	0.08***
Sonority dissonance	-0.43***	0.12***	-0.28***	0.30***	0.47***

The secondary goal was to explore the influence of a number of rhythmic and pitch features on listeners’ perceived emotions. A summary of standardized coefficients is shown in Table 7. Some of our findings are novel and open avenues for investigation on the role of rhythmic structure on perceived emotion, especially in multi-part music. Most notably, composite nPVI and nPVI group difference, which provide measures of note-to-note duration variability in rhythmic sequences overall and between rhythmic groups, were among the strongest predictors of participants’ ratings for all five emotional dimensions. These two factors were predictive of lower mood, energy, and movement ratings, while their effect on perceived dissonance and tension were in opposite directions, with composite nPVI being predictive of lower, and nPVI group difference higher, participants’ ratings. These observations are consistent with findings that higher levels of rhythmic complexity have a negative effect on entrainment, which may result in reduced enjoyment (Labbé & Grandjean, 2014), especially given that less regular rhythms have been found to be perceived as less pleasant (Scherer & Zentner, 2001). On the other hand, the observed negative effect of rhythmic complexity on mood are not consistent with earlier findings of complex rhythms being linked with perceived happiness (Scherer & Oshinski, 1977), which could be explained by the lack of a consensus in how rhythmic complexity is defined (i.e., “even” vs. “uneven” as compared to computational measures based on metric fit or duration variability). It is also possible that the effect of rhythmic complexity on perceived emotions is not linear, which could be further tested using excerpts that offer a range of values along one or more complexity measures. The varied effects of the nPVI measures on perceived dissonance and tension in polyphonic texture warrant further investigation.

The effects of the three other rhythmic factors, composite event density, event density ratio, and nested ratio were less consistent and/or relatively smaller, but not negligible. Composite event density, which provides a measure of rate presentation and a proxy for tempo, was most notably positively correlated with energy and tension, which is consistent with findings that link shorter note values and faster tempi with higher arousal and perceived tension (Balkwill & Thompson, 1999; Fernández-Sotos et al., 2016). Higher event density ratio, which corresponds to a texture in which the densities of the two rhythmic groups are similar, was positively correlated with mood and movement, and negatively correlated with dissonance and tension. In other words, more homogeneous rhythmic structure contributed to a more positive judgment of mood and tendency to move along the musical excerpt, as well as lower perceived dissonance and tension. Nested ratio, which measures the degree of coincidence between parts in terms of event onsets, was also negatively correlated with dissonance and tension, and to a lesser extent, energy, suggesting that excerpts with a texture that is more rhythmically integrated are perceived as conveying lower energy, dissonance, and tension. Given that these measures were designed to address the specific features of rhythmic structure in polyphonic textures, these results call for further research.

Among pitch features, sonority dissonance and pitch mean had a noticeably larger effect on several emotional dimensions. The positive correlation between sonority dissonance and perceived dissonance and tension was expected. The negative effect of dissonance on mood (Costa et al., 2004; ​​Dellacherie et al., 2010; Fritz et al., 2013; Kawakami et al., 2013) and the positive effect on arousal (Dellacherie et al., 2010; Gomez & Danuser, 2007) are also well-documented. On the other hand, the negative correlation with movement calls for further investigation. Recent research suggests that along with rhythmic complexity, harmonic complexity may also follow the inverted-U effect (Berlyne, 1971; Matthews et al., 2019), which, given that sonority dissonance is primarily based on harmonic intervals, is consistent with our findings. However, the relationship of sonority dissonance with entrainment has received little attention. Finally, the effects of pitch mean were also relatively strong, while those of pitch range were less consistent and generally smaller. Higher pitch mean was predictive of lower mood, energy, and movement ratings, and higher ratings of perceived dissonance and tension, while wider pitch range was predictive of higher ratings for all rated emotional dimensions, except for mood, and most notably for perceived energy. These results somewhat contrast with studies that associate high-pitched music with positive emotion descriptions (Aljanaki et al., 2014), which could be explained by interaction with rhythmic features (Schellenberg et al., 2000), a link that requires investigation beyond the present study.

EXPERIMENT II: PITCH PRESENTATION

We aimed to test the potential influence of common methods used in the preparation of musical stimuli for experiments that investigate listeners’ perceived emotions while listening to music. In Experiment 1, we tested the influence of production mode as a method to generate musical stimuli. In Experiment 2, we addressed the potential influence of randomization methods.

The primary goal of the second experiment was to determine whether pitch presentation (ordered vs. randomized) had an effect on listeners’ judgment of five emotional dimensions (mood, energy, movement, dissonance, and tension), and thus address the gap in research on the effect of pitch structure on perceived emotions. As in the first experiment, a secondary goal was the exploratory examination of musical features and their possible effects on participants’ emotion ratings. The musical features were the same as those used previously. Because of the significant effects of instrumentation observed in Experiment 1, and because realistic synthesis of a string quartet was difficult to achieve, we limited our musical excerpts to those originally composed for solo piano.

Similar to the first experiment, we assumed a null hypothesis for the influence of pitch presentation on listeners’ ratings of the five emotional dimensions. For musical features, we informally expected event density to be positively correlated with mood and movement, and nested ratio to be negatively correlated with energy. Based on previous research on the influence of pitch structure on valence and arousal, we also expected a negative correlation of pitch mean with mood and energy, and sonority dissonance to be predictive of perceived dissonance and tension.

Methods

PARTICIPANTS

Experiment 2 was conducted in the same manner as in the first experiment. One hundred and sixty-one participants completed the study.[7] One participant who reported having difficulty with English in everyday life and one participant who reported having previously completed the survey were excluded from the sample; the results reported here are those associated with the responses of the remaining 159 participants. Most participants self-identified as women (77%) as compared to men (18%) and non-binary persons (2%); four participants chose not to report their gender. Participants’ ages ranged from 18 to 48 (M = 22.03; SD = 5.27). Participants’ self-reported formal musical training ranged from 0 to 20 years (M = 5.40; SD = 4.64). In a post-survey questionnaire, most of the participants (42%) reported being unfamiliar with the musical style represented by the excerpts used in the survey; the remaining participants were either familiar (31%) or “not sure” (26%). Most participants (73%) reported being unfamiliar with a specific excerpt either through listening or performance, with a smaller proportion being familiar with at least one excerpt (14%) or “not sure” (12%). Finally, most participants reported using their computer’s built-in speakers (45%) and noise canceling (26%) or standard (21%) headphones or earbuds; the remaining eleven participants used external speakers.

MATERIALS

We employed the subset of eight excerpts for solo piano from Experiment 1 in this experiment (presented in Table 1). These excerpts were written by five different composers (Béla Bartók, Manuel de Falla, George Gershwin, Frank Martin, and Sergei Prokofiev) and ranged from 1914 to 1965 in terms of composition year. The duration of the musical excerpts ranged from about 5 to 9 s (M = 7.29; SD = 1.06). Given the variety of polyrhythmic textures represented by the selected examples, the small sample size was determined to be sufficient as a testing set for our measures.

Each excerpt was rendered into two versions: one with pitches unchanged from the original (“ordered”) and one with pitches assigned randomly while preserving the original rhythmic structure (“randomized”).[8] To prepare the randomized version of each excerpt, we determined the highest and lowest pitches, and the specific set of octave-equivalent pitches (i.e., pitch classes) used in the excerpt. Pitch was randomized so that the same pitch classes were presented within the same overall pitch range as in the original version, but not necessarily the same octave. To prevent excessive pitch repetition, a note was never assigned the same pitch as an immediately preceding note onset, whether that pitch occurred as an individual note or within a chord. Both versions were rendered with the EastWest audio library Steinway D virtual instrument.

PROCEDURE AND DESIGN

The procedure and design for Experiment 2 was the same as that in Experiment 1, except for the experimental stimuli the participants listened to. Participants completed two practice trials, with ordered or pitch-randomized excerpts randomly assigned for each practice trial; these trials were based on different source materials and were not included in the analysis. Excerpt and pitch presentation were within-subjects variables. The order of the excerpts was randomized for each participant; each participant also heard only one version of each of the eight excerpts (ordered or randomized), which was randomly selected for that participant.

MEASURES

In addition to participants’ ratings of five dependent variables using seven-point Likert scales, we used the same nine pitch and rhythm features to explore the relationship between musical features and participants’ perceived emotions (see Table 3). Figure 2 illustrates the visual analytic markup for a sample (“ordered”) excerpt used in the experiment. Note that although piano excerpts are notated on two staves, each stave can include more than one part. The sample excerpt features two contrasting rhythmic groups (A and B), with the lower part assigned to Group A by default. Group A notes are colored in red, while Group B notes are in blue. The top two staves are the original score and underneath are the extracted rhythmic sequences for Group A only, Group B only, Groups A and B combined (“composite”), and the intersection of Groups A and B (“coincidence”). In this excerpt, the polyrhythm results from the triple grouping of the sixteenth notes in Group B against the regular duple subdivision of the 2/4 measure (represented by Group A).

Each of the eight excerpts used as source materials feature polyrhythmic activity that varies in terms of design as well as complexity across the excerpts. The extracted rhythmic measures are shown in Table 8, with those for R350 (Figure 2) highlighted in gray. The values in the first three columns are based on the full texture. The composite event density of R350 is about 6 events per seconds, a value that is close to the average rate of presentation in this set of excerpts, while its event density ratio is the highest, i.e., the event densities of the two groups are most similar. As it includes many successive sixteenth notes, the composite nPVI is somewhat lower than the average, while the nPVI group difference is relatively high due to the contrast in note-to-note regularity introduced by the sixteenth rests in Group B. Finally, the nested ratio shows a relatively low degree of integration, which is characteristic of polyrhythmic textures, although the value is slightly higher than the average for this set.

Table 8. Rhythm measures for experimental excerpts in Experiment 2, with means and standard deviations in parentheses. Rhythmic structure is invariable across ordered and randomized versions.

Excerpt	Composite Event Density	Composite nPVI	Nested Ratio	Event Density Ratio	nPVI Group Difference
R310	9.29	12.40	0.10	0.17	36.37
R324	2.89	45.56	0.44	0.57	55.23
R350	6.27	18.18	0.33	0.88	62.37
R534x	3.02	35.20	0.37	0.76	62.03
R560	6.60	0	0.33	0.33	57.49
R574	10.10	42.01	0.17	0.40	100
R582	5.85	0	0	0.56	43.53
R686x	5.81	8.00	0.47	0.44	21.33
Mean (SD)	6.23 (2.57)	20.17 (18.42)	0.28 (0.17)	0.51 (0.23)	54.79 (23.14)

Figure 3 presents a transcription of the randomized variant of the same excerpt. As shown by the analytic markup and the composite rhythms below the score, the underlying rhythmic structure is the same. These two examples illustrate how pitch register plays a role in the perceptual parsing of rhythmic layers. Despite preserving the sequence of rhythmic values from the original excerpt, the randomization of pitch classes (rather than specific pitches) significantly hides the rhythmic design of the multi-part texture. Thus, we did not discard the possibility that structural features related to the divided groups could have an effect on participants’ perceived emotions, and all five rhythm measures, as well as audio file duration were considered in the comparison of models. Additionally, three pitch measures were also considered. These measures are derived from pitch data extracted from both excerpt versions (ordered and randomized). Register measures were excluded from the analysis because they are not independent. Pitch measures data are shown in Table 9, with the pitch data for the excerpt shown as Figure 3 (R350 “randomized”) highlighted in gray.

ANALYSIS

The same approach was taken as in Experiment 1, with the exception that we did not consider models with only pitch or rhythm factors. Audio file duration, included in Experiment 1, was removed in the final models for Experiment 2 due to high VIF values (> 50). Pitch range, also included in Experiment 1, was removed due to collinearity.

Table 9. Pitch measures for ordered and randomized excerpts, with overall mean and standard deviations in parentheses. Measures for ordered excerpts are shown on top with measures for randomized excerpts below; register and pitch range measures are invariable across ordered and randomized versions of the same excerpt.

Excerpt	Low Register	High Register	Pitch Range	Pitch Mean	Sonority Dissonance
R310	47	64	18	58.21 54.50	1 3.57
R324	57	79	23	69.05 67.77	2.01 3.04
R350	34	84	51	58.33 58	4.38 3.79
R534x	62	81	20	69.36 70.81	3.14 3.76
R560	41	83	43	57.47 61.81	2.17 2.37
R574	30	66	37	39.52 47.24	1.75 2.28
R582	34	68	35	51.57 49.63	2.60 4.04
R686x	32	93	62	63.96 63.84	2.36 3.60
Mean (SD)	42.10 (11.68)	77.30 (9.88)	36.10 (15.06)	58.82 (8.81)	2.87 (0.94)

RESULTS II

Participants rated eight excerpts on five seven-point Likert scales (N = 1,272). The average rating for energy was the highest (M = 4.41; SD = 1.59), while the average rating for movement was the lowest (M = 3.19; SD = 1.76). The average ratings were 3.85 (SD = 1.47) for mood, 3.90 (SD = 1.57) for tension, and 4.10 (SD = 1.72) for dissonance.

Effects of Pitch Presentation

To test the effect of pitch presentation (ordered vs. randomized) on the five dependent variables, we conducted a one-way MANOVA with pitch presentation as the independent variable and the five dependent variables combined. The effect of pitch presentation was statistically significant and large, F(1, 1270) = 66.20, p < .001. One-way ANOVAs were used to identify which of the dependent variables had statistically significant differences between the two groups of the independent variable. The differences in ratings for each of the five dependent variables were also statistically significant at the p < .001 level.

Point biserial correlations were calculated between the two pitch presentation conditions (ordered = 1; randomized = 2) and each of the five dependent variables; all correlations were significant at the p < .001 level. Pitch presentation was negatively correlated with mood (rpb = -.37), energy (rpb = -.14), and movement (rpb = -.33). Pitch presentation was also positively correlated with dissonance (rpb = .33) and tension (rpb = .17).

Effects of Musical Features

The model with all factors provided a better fit, with about 10% (dissonance) to 44% (energy) of the variance accounted for (see Table 10). While the VIF values for composite event density and pitch mean are relatively high (see Table 11), we felt that they were not exceptionally high enough to warrant exclusion.

Table 10. Comparison of model fits using Akaike Information Criterion (AIC) and Bayesian Information Criterion (BIC) with the all-factors model (7 factors) and the rhythm-only model (5 factors).

Emotion	Model	df	AIC	BIC	Adjusted R2	p value
Mood	All	1264	353	400	.239	< .001
	Rhythm	1266	545	582	.114	< .001
Energy	All	1264	134	180	.440	< .001
	Rhythm	1266	183	219	.417	< .001
Movement	All	1264	1398	1444	.109	< .001
	Rhythm	1266	1503	1539	.031	< .001
Dissonance	All	1264	1063	1109	.098	< .001
	Rhythm	1266	1152	1188	.031	< .001
Tension	All	1264	885	932	.132	< .001
	Rhythm	1266	946	982	.088	< .001

Table 11. Variance Inflation Factor (VIF) values for the all-factors model (7 factors) and rhythm-only model (5 factors). Values in the full model are all below 7; values in the rhythm-only model are below 4.

Factor	VIF
	All Factors	Rhythm Only
Composite event density	5.38	3.11
Composite nPVI	2.51	2.36
Event density ratio	3.83	2.16
nPVI group difference	3.77	2.86
Nested ratio	2.62	1.64
Pitch mean	6.82	–
Sonority dissonance	2.08	–

Table 12. Parameter estimates and standardized coefficients for rhythmic and pitch features for participants’ ratings of mood, energy, mood, movement, dissonance, and tension.

Emotional Dimension	Parameter	Estimate	Standardized Coefficient	St. Error	t value	Pr(>|t|)
Mood (adjusted R2 = .239)	(Intercept)	1.72	n/a	0.18	9.30	< .001
	Composite event density	0.05	0.41	0.01	7.15	< .001
	Composite nPVI	0.002	0.11	0.001	2.74	0.006
	Event density ratio	0.56	0.38	0.07	7.84	< .001
	nPVI group difference	-0.008	-0.54	0.001	-11.33	< .001
	Nested ratio	-0.06	-0.03	0.08	-0.80	0.42
	Pitch mean	0.002	0.06	0.002	0.91	0.36
	Sonority dissonance	-0.18	-0.51	0.01	-14.47	< .001
Energy (adjusted R2 = .440)	(Intercept)	1.60	n/a	0.17	9.49	< .001
	Composite event density	0.09	0.66	0.01	13.63	< .001
	Composite nPVI	-0.007	-0.34	0.001	-10.32	< .001
	Event density ratio	0.64	0.40	0.06	9.84	< .001
	nPVI group difference	-0.005	-0.29	0.001	-7.12	< .001
	Nested ratio	0.74	0.34	0.07	10.09	< .001
	Pitch mean	-0.01	-0.21	0.002	-3.76	< .001
	Sonority dissonance	-0.07	-0.20	0.01	-6.44	< .001
Movement (adjusted R2 = .109)	(Intercept)	2.11	n/a	0.28	7.59	< .001
	Composite event density	0.03	0.16	0.01	2.64	0.01
	Composite nPVI	0.0001	0.004	0.001	0.09	0.93
	Event density ratio	0.72	0.35	0.11	6.74	< .001
	nPVI group difference	-0.01	-0.38	0.001	-7.43	< .001
	Nested ratio	0.15	0.05	0.12	1.24	0.22
	Pitch mean	-0.01	-0.12	0.004	-1.76	0.08
	Sonority dissonance	-0.20	-0.40	0.02	-10.52	< .001
Dissonance (adjusted R2 = .098)	(Intercept)	0.51	n/a	0.24	2.11	0.03
	Composite event density	-0.003	-0.02	0.01	-0.30	0.76
	Composite nPVI	-0.005	-0.22	0.001	-5.28	< .001
	Event density ratio	-0.36	-0.20	0.09	-3.86	< .001
	nPVI group difference	0.01	0.38	0.001	7.26	< .001
	Nested ratio	0.09	0.04	0.11	0.88	0.38
	Pitch mean	0.01	0.19	0.003	2.74	0.01
	Sonority dissonance	0.16	0.36	0.02	9.49	< .001
Tension (adjusted R2 = .132)	(Intercept)	1.56	n/a	0.23	6.87	< .001
	Composite event density	-0.01	-0.09	0.01	-1.44	0.15
	Composite nPVI	-0.01	-0.29	0.001	-6.93	< .001
	Event density ratio	-0.18	-0.11	0.09	-2.11	0.04
	nPVI group difference	0.01	0.34	0.001	6.68	< .001
	Nested ratio	0.42	0.18	0.10	4.30	< .001
	Pitch mean	-0.01	-0.20	0.003	-2.86	0.004
	Sonority dissonance	0.12	0.28	0.02	7.52	< .001

Energy was the variable for which variance was most consistently predicted by the model (adjusted R² = .440), while dissonance was the least (adjusted R² = .098). Parameter estimates and standardized coefficients for rhythmic and pitch features for each of the five emotional dimensions are presented in Table 12. Standardized coefficients are used below to interpret the relative strength of the factors on participants’ ratings.

Five of the seven factors had a statistically significant effect on mood, accounting for 24% of variance in participants’ ratings. Composite event density, event density ratio, and composite nPVI were positively correlated with mood, while nPVI group difference and sonority dissonance, which had the strongest effect, were negatively correlated. In contrast, all seven factors were found to be statistically significant for participants’ ratings of perceived energy, with composite event density and event density ratio being the most reliable predictors of higher perceived energy. Nested ratio was also positively correlated with participants’ perceived energy, while composite nPVI, nPVI group difference, pitch mean, and sonority dissonance were negatively correlated. Four of the seven factors were found to have a statistically significant effect on participants’ ratings of perceived movement, accounting for 11% of variance. Event density ratio and composite event density were positively correlated with participants’ ratings of their tendency to move along the music. On the other hand, sonority dissonance, nPVI group difference, and pitch mean were negatively correlated with perceived movement.

Results for participants’ ratings of perceived dissonance and tension were very similar, with the exception of the effects of pitch mean and nested ratio. Sonority dissonance and nPVI group difference were relatively strong predictors and positively correlated with both dissonance and tension ratings, while composite nPVI and event density ratio were negatively correlated. On the other hand, pitch mean was positively correlated with perceived dissonance, but negatively correlated with perceived tension. Finally, nested ratio was positively correlated with tension ratings, but its effect on dissonance was not statistically significant. Composite event density was not statistically significant for these dimensions.

DISCUSSION II

The primary goal of Experiment 2 was to investigate the influence of pitch presentation on listeners’ perceived emotions, within the context of rhythmically and harmonically complex music excerpts for piano. While results have identified some trends in how pitch presentation influences listeners’ experience, the findings cannot be easily mapped onto existing research. Instead, our study introduces a novel approach to investigate questions pertaining to musical expression using naturalistic stimuli. The finding that pitch presentation has a main effect on listeners’ perceived emotions is consistent with previous studies that observed a relationship between melodic contour and perceived valence (Gomez & Danuser, 2007; Juslin, 2019). More importantly, our results point to more global effects of pitch structure on specific emotional dimensions. The finding that excerpts in which pitch presentation followed the original composition order were associated with a more positive mood, higher energy, and a greater impulse to move along with the music suggests that the specific decisions made by the composer with regards to pitch structure contribute to listeners’ enjoyment of the music, even in complex and unfamiliar music. Furthermore, the finding that randomized excerpts with the same underlying, although somewhat less palpable, rhythmic structure and pitch class content were perceived as more dissonant and with more tension is consistent with the higher sonority dissonance observed in seven out of the eight pitch-randomized excerpts as compared with their ordered counterparts (Table 9). This points to the importance of small-scale interactions between pitch and duration, including octave specific relationships. These results suggest that when using pitch randomization to generate musical stimuli, more attention needs to be devoted to reporting the degree of dissonance, using measures that allow for generalization of findings.

Table 13. Summary of standardized coefficients for seven rhythm and pitch factors for each of the rated emotional dimensions in Experiment 2, with significance level for each. Significance levels are: * p < .05, ** p < .01, *** p < .001. Event density ratio, nPVI group difference, and sonority dissonance are significant for all emotional dimensions, with nPVI group difference being the strongest predictor for three out of five.

Parameter		Standardized coefficient
	Mood	Energy	Movement	Dissonance	Tension
Composite event density	0.41***	0.66***	0.16**	-0.02	-0.09
Composite nPVI	0.11**	-0.34***	0.004	-0.22***	-0.29***
Event density ratio	0.38***	0.40***	0.35***	-0.20***	-0.11*
nPVI group difference	-0.54***	-0.29***	-0.38***	0.38***	0.34***
Nested ratio	-0.03	0.34***	0.05	0.04	0.18***
Pitch mean	0.06	-0.21***	-0.12	0.19**	-0.20**
Sonority dissonance	-0.51***	-0.20***	-0.40***	0.36***	0.28***

The secondary goal was to explore the influence of a number of rhythmic and pitch features on listeners’ perceived emotions. A summary of standardized coefficients is shown in Table 13. Because the measures were extracted from both ordered and randomized excerpts, the findings related to these features further qualify the observed effects of pitch presentation and attest to the importance of considering the specific structural features of the music used as source materials for experimental stimuli. In the current study, we used rhythmic and pitch measures pertaining to the whole texture (composite event density, composite nPVI, pitch mean, and sonority dissonance) as well as rhythmic measures derived from the interaction of composite rhythms from the divided texture (event density ratio, nPVI group difference, and nested ratio). Several features were observed to be reliable predictors of listeners’ perceived emotions, with the most variance accounted for in perceived energy ratings. The strongest predictors overall were event density ratio, nPVI group difference, and sonority dissonance. Event density ratio was predictive of higher mood, energy, and movement ratings, suggesting that a more homogeneous texture in terms of rate of presentation of events in different parts has a positive effect on valence, arousal, as well as the impulse to move along with the music, but a depressing effect on perceived dissonance and tension, effects that are novel and remain to be further investigated. On the other hand, nPVI group difference, which is indicative of a texture in which one rhythmic group is more irregular than the other, was found to be predictive of lower ratings in perceived mood, energy, and movement, but higher ratings in terms of perceived dissonance and tension. As expected, sonority dissonance had a negative effect on perceived mood as well as being predictive of higher ratings of dissonance and tension, but the observed negative effects on perceived energy and the impulse to move along with the music are somewhat unexpected. Previous research points to an inverted-U relationship between rhythmic complexity and entrainment (Stupacher et al., 2022). However, apart from findings related to a link between harmonic complexity and the pleasure associated with entrainment (Matthews et al., 2019), evidence of an influence of sonority dissonance on sensorimotor synchronization is limited.

Other factors had a smaller and/or more limited effect on perceived emotions, although they offer additional insight into the effect of these parameters on perceived emotions from the perspective of multi-part music. For example, pitch mean was found to have a significant effect on energy, dissonance, and tension ratings, but not mood or movement. Research on the effect of pitch height on valence and arousal is inconclusive, and effects have been found to vary based on gender (Jacquet et al., 2014). In our study, higher pitch mean did not have a significant effect on mood, but was correlated with lower energy and tension ratings. This is consistent with previously observed relationships of pitch mean with low arousal emotion words such as tenderness, nostalgia, and calmness (Aljanaki et al., 2014), but inconsistent with findings that low register melodies correlate with higher tension (Granot & Eitan, 2011) or that harmonic intervals presented in a higher register are associated with higher perceived tension (Costa & Nese, 2020). This suggests that further research using different types of textures and more specific emotion descriptors may be needed to elucidate these contradictions. The positive effect of composite event ratio (a proxy for tempo) on mood and energy is consistent with research that associates positively valenced and higher arousal emotion words with faster tempo (Juslin & Sloboda, 2001) and provides evidence that these effects also apply to complex polyphonic music. Finally, the observed significant positive effect of the number of coinciding event onsets (nested ratio) on perceived energy and tension is novel and requires further investigation using more controlled stimuli.

CONCLUSION

Several of our findings show that the interaction of rhythmic sequences in multi-part music plays a significant role in listeners’ emotional experience, which calls into question the ecological validity of studies that focus on relatively simple musical sequences to study perceived emotions. Specifically, findings based on rhythmic features suggest that more attention should be devoted to finer-grain musical features and the interaction between parts. Some of the factors were found to have very small, yet significant effects on participants’ ratings of mood, energy, movement, dissonance, and tension. These factors may require investigation using more controlled experiments. However, these results may also point to compositional practice. In creating effects, composers rarely manipulate a single parameter. Rather, they use a combination of parameters, which is suggested by the observed covariance of subjective ratings of perceptual features in ecological stimuli (Friberg et al., 2014) and the interactive nature of the effects associated with parameters, such as loudness and register (e.g., Granot & Eitan, 2011). Thus, although some factors, such as pitch range, may show stronger effects in controlled experiments, the results may be difficult to generalize to actual musical compositions. In this study, we avoided including factors with high collinearity and aimed to keep variance low, which puts in relief the predictive reliability of parameters such as nPVI group difference and sonority dissonance in listeners’ perception of musical emotions.

Taken together, these results trace an attractive path of investigation into the influence of musical features in multi-part music. Approaches that favor analysis of multiple factors in the sampled music may help uncover the effect of different parameters. Further analysis may also help identify combinations of parameters used by composers to create specific effects beyond the scope of the current study. The music-analytical approach taken in our study can also inform developing research in the implementation of Web-based digital audio applications that focus on the communication of emotion through musical compositions (Turchet et al., 2020). Although the source materials in this study belong to a specific geographical and historical style, the use of composite rhythms as a basis for computational measures is not style specific. Thus, our results could be used to advance automatic music information retrieval focused on emotion. The Suter (1980) Corpus, from which the study excerpts were selected, includes a wide range of examples extracted from instrumental, vocal, and stage works by twenty composers written for solo keyboard instruments as well as for ensembles of different sizes, from 2 to 30 parts. Future studies could include more substantial selections of multi-part music from this corpus as well as from a wider range of musical cultures, including compositions for different types of ensembles that feature a variety of textural strategies. The influence of the interaction of the different parts in the musical texture on perceived emotion could also be further examined, for example the influence of the relative density and regularity of different rhythmic layers and the degree of integration between layers. Such materials could be explored from an interdisciplinary perspective that includes machine learning and online studies with human participants, which has the potential to enrich and diversify user-based music streaming.

ACKNOWLEDGMENTS

The authors thank the members of the UBC Rhythm Computation and Cognition Lab for the encoding of the musical excerpts, Walker Williams for audio preparation, and Daniel Shanahan for statistical consulting. We also thank the editor for helpful suggestions on data analysis. This research is supported in part by funding from the Social Sciences and Humanities Research Council of Canada (GR012182). This article was copyedited by Jonathan Tang and layout/edited by Niels Chr. Hansen.

REFERENCES

Aljanaki, A., Wiering, F., & Veltkamp, R. C. (2014). Computational modeling of induced emotion using GEMS. In Proceedings of the 15th Conference of the International Society for Music Information Retrieval (pp. 373–378). https://archives.ismir.net/ismir2014/paper/000325.pdf

Bachorik, J. P., Bangert, M., Loui P., Larke, K., Berger, J., Rowe, R., & Schlaug, G. (2009). Emotion in motion: investigating the time-course of emotional judgments of musical stimuli. Music Perception, 26(4), 355–364. https://doi.org/10.1525/mp.2009.26.4.355

Balkwill, L.-L., & Thompson, W. F. (1999). A cross-cultural investigation of the perception of emotion in music: Psychophysical and cultural cues. Music Perception, 17(1), 43–64. https://doi.org/10.2307/40285811

Berlyn, D. E. (1971). Aesthetics and Psychobiology. Appleton-Century-Crofts.

Bhatara, A., Tirovolas, A. K., Duan, L. M., Levy, B., & Levitin, D. J. (2011). Perception of emotional expression in musical performance. Journal of Experimental Psychology: Human Perception and Performance, 37(3), 921–934. https://doi.org/10.1037/a0021922

Costa, M., Fine, P., & Ricci Bitti, P. E. (2004). Interval distributions, mode, and tonal strength of melodies as predictors of perceived emotion. Music Perception, 26(1), 1–14. https://doi.org/10.1525/mp.2004.22.1.1

Costa, M., & Nese, M. (2020). Perceived tension, movement, and pleasantness in harmonic musical intervals and noises. Music Perception, 37(4), 298–322. https://doi.org/10.1525/mp.2020.37.4.298

Daynes, H. (2011). Listeners’ perceptual and emotional responses to tonal and atonal music. Psychology of Music, 39(4), 468–502. https://doi.org/10.1177/0305735610378182

Dellacherie, D., Roy, M., Hugueville, L., Peretz, I., & Samson, S. (2011). The effect of musical experience on emotional self‐reports and psychophysiological responses to dissonance. Psychophysiology, 48(3), 337–349. https://doi.org/10.1111/j.1469-8986.2010.01075.x

Drake, C., Penel, A., & Bigand, E. (2000). Tapping in time with mechanically and expressively performed music. Music Perception, 18(1), 1–23. https://doi.org/10.2307/40285899

Eerola, T., Friberg, A., & Bresin, R. (2013). Emotions expression in music: contribution, linearity, and additivity of primary musical cues. Frontiers in Psychology, 4, 487. https://doi.org/10.3389/fpsyg.2013.00487

Eerola, T., & Vuoskoski, J. K. (2013). A review of music and emotion studies: approaches, emotion models, and stimuli. Music Perception, 30(3), 307–340. https://doi.org/10.1525/mp.2012.30.3.307

Fernández-Sotos, A., Fernández-Caballero, A., & Latorre, J. M. (2016). Influence of tempo and rhythmic unity in musical emotion regulation. Frontiers in Computational Neuroscience, 10, 80. https:/doi.org/10.3389/fncom.2016.00080

Farbood, M. M., & Price, K. (2014). Timbral features contributing to perceived auditory and musical tension. In M. K. Song (Ed.), Proceedings of the 13th International Conference for Music Perception and Cognition, (pp. 21–26). https://www.academia.edu/23178084/Timbral_Features_Contributing_to_Perceived_Auditory_and_Musical_Tension

Fox, J., & Weisberg, S. (2019). An R Companion to Applied Regression (3rd Ed.). Sage Publications.

Friberg, A., Schoonderwaldt, E., Hedblad, A., Fabiani, M., & Elowsson, A. (2014). Using listener-based perceptual features as intermediate representations in music information retrieval. Journal of the Acoustical Society of America, 136(4), 1951–1963. https://doi.org/10.1121/1.4892767

Fritz, T. H., Renders, W., Müller, K., Schmude, P., Leman, M., Turner, R., & Villringer, A. (2013). Anatomical differences in the human inferior colliculus relate to the perceived valence of musical consonance and dissonance. European Journal of Neuroscience, 38(7), 3099–3105. https://doi.org/10.1111/ejn.12305

Gabrielson, A., & Juslin, P. N. (1996). Emotional expression in music performance: Between the performer’s intention and the listener’s experience. Psychology of Music 24(1), 68–91. https://doi.org/10.1177/0305735696241007

Gagnon, L., & Peretz, I. (2003). Mode and tempo relative contributions to “happy-sad” judgements in equitone melodies. Cognition and Emotion, 17(1), 25–40. https://doi.org/10.1080/02699930302279

Gilburt, S. J. M., Chau, C.-j., & Horner, A. (2016). The effects of pitch and dynamics on the emotional characteristics of bowed string instruments. In Proceedings of the 42nd International Computer Music Conference (pp. 405–410). https://www.semanticscholar.org/paper/The-Effects-of-Pitch-and-Dynamics-on-the-Emotional-Gilburt-Chau/7009b31a26479b5059b50734e0569c4169d83590?utm_source=direct_link

Gomez, P., & Danuser, B. (2007). Relationships between musical structure and psychophysiological measures of emotion. Emotion, 7(2), 377–387. https://doi.org/10.1037/1528-3542.7.2.377

Granot, R. Y., & Eitan, Z. (2011). Musical tension and the interaction of dynamic auditory parameters. Music Perception, 28(3), 219–245. https://doi.org/10.1525/mp.2011.28.3.219

Hähnel, T., & Berndt, A. (2010). Synthetic and pseudo-synthetic music performances: an evaluation. In Proceedings of the 3^rd International Conference of Students of Systematic Musicology, Cambridge, UK, September 13–15, 2010. Cambridge. https://www.researchgate.net/publication/261804611_Synthetic_and_Pseudo-Synthetic_Music_Performances_An_Evaluation

Hailstone, J. C., Omar, R., Henley, S. M. D., Frost, C., Kenward, M. G., & Warren, J. D. (2009). It’s not what you play, it’s how you play it: timbre affects perception of emotion in music. The Quarterly Journal of Experimental Psychology, 62(11), 2141–2155. https://doi.org/10.1080/17470210902765957

Handel, S., & Lawson, G. R. (1983). The contextual nature of rhythmic interpretation. Perception & Psychophysics, 34(2), 103–120. https://doi.org/10.3758/BF03211335

Handel, S., & Oshinsky, J. S. (1981). The meter of syncopated auditory polyrhythms. Perception & Psychophysics, 30(1), 1–9. https://doi.org/10.3758/BF03206130

Hevner, K. (1937). The affective value of pitch and tempo in music. The American Journal of Psychology, 49(4), 621–630. https://doi.org/10.2307/1416385

Honing, H. (2007). Is expressive timing relational invariant under tempo transformation? Psychology of Music, 35(2), 276–285. https://doi.org/10.1177/0305735607070380

Jacquet, L., Danuser, B., & Gomez, P. (2014). Music and felt emotions: how systematic pitch level variations affect the experience of pleasantness and arousal. Psychology of Music, 42(1), 51–70. https://doi.org/10.1177/0305735612456583

Jones, M. R. (2010). Music perception: current research and future directions. In M. R. Jones, R. Fay, and A. Popper (Eds.), Music Perception (pp. 1–17). Springer. https://doi.org/10.1007/978-1-4419-6114-3_1

Juslin, P. N. (1997). Perceived emotional expression in synthesized performances of a short melody: capturing the listener’s judgment policy. Musicae Scientiae, 1(2), 225–256. https://doi.org/10.1177/102986499700100205

Juslin, P. N. (2019). Musical Emotions Explained: Unlocking the Secrets of Musical Affect. Oxford University Press. https://doi.org/10.1093/oso/9780198753421.001.0001

Juslin, P. N., & Lindström, E. (2010). Musical expression of emotions: modeling listeners’ judgements of composed and performed features. Music Analysis, 29(1–3), 334–364. https://doi.org/10.1111/j.1468-2249.2011.00323.x

Juslin, P. N., & Sloboda, J. A. (2001). Music and Emotion: Theory and Research. Oxford University Press. https://doi.org/10.1093/oso/9780192631886.001.0001

Kamenetsky, S. B., Hill, D. S., & Trehub, S. E. (1997). Effect of tempo and dynamics on the perception of emotion in music. Psychology of Music 25(2), 149–160. https://doi.org/10.1177/0305735697252005

Kassambara, A. (2022). rstatix: Pipe-Friendly Framework for Basic Statistical Tests. In R package version 0.7.1 [Online]. https://CRAN.R-project.org/package=rstatix

Kawakami, A., Furukawa, K., Katahira, K., Kamiyama, K., & Okanoya, K. (2013). Relations between musical structures and perceived and felt emotions. Music Perception, 30(4), 407–417. https://doi.org/10.1525/mp.2013.30.4.407

Kendall, R. A., Carterette, E. C., & Hajda, J. M. (1999). Perceptual and acoustical features of natural and synthetic orchestral instrument tones. Music Perception, 16(3), 327–363. https://doi.org/10.2307/40285796

Kirke, A., & Miranda, E. R. (2009). A survey of computer systems for expressive music performance. ACM Computing Surveys, 42(1), 3, 1–41. https://doi.org/10.1145/1592451.1592454

Krumhansl, C. L. (2002). Music: a link between cognition and emotion. Current Directions in Psychological Science, 11(2), 45–50. https://doi.org/10.1111/1467-8721.00165

Labbé, C., & Grandjean, D. (2014). Musical emotions predicted by feelings of entrainment. Music Perception, 32(2), 170–185. https://doi.org/10.1525/mp.2014.32.2.170

Le Groux, S. & Verschure, P. F. M. J. (2010). Emotional responses to the perceptual dimensions of timbre: a pilot study using physically informed sound synthesis. In Proceedings of the 7th International Symposium on Computer Music Modeling (pp. 1–15). https://www.academia.edu/23454127/Emotional_Responses_to_the_Perceptual_Dimensions_of_Timbre_A_Pilot_Study_Using_Physically_Informed_Sound_Synthesis

Lehne, M. (2014). Emotional Experiences of Tension and Suspense: Psychological Mechanisms and Neural Correlates. Unpublished doctoral dissertation, Freie Universität, Berlin. Retrieved from https://d-nb.info/1054637164/34

Matthews, T. E., Witek, M. A. G., Heggli, O. A., Penhune, V. B., & Vuust, P. (2019). The sensation of groove is affected by the interaction of rhythmic and harmonic complexity. PLoS ONE, 14(1), e0204539. https://doi.org/10.1371/journal.pone.0204539

Oore, S., Simon, I., Dielemen, S., Eck, & D. Simonyan, K. (2020). The time with feeling: learning expressive musical performance. Neural Computing and Applications, 32, 955–967. https://doi.org/10.1007/s00521-018-3758-9

Patel, A. D., & Daniele, J. R. (2003). An empirical comparison of rhythm in language and music. Cognition, 87(1), B35–B45. https://doi.org/10.1016/S0010-0277(02)00187-7

Peretz, I., Gosselin, N., Belin, P., Zatorre, R. J., Plailly, J., & Tillmann, B. (2009). Music lexical networks: the cortical organization of music recognition. Annals of the New York Academy of Sciences, 1169(1), 256–265. https://doi.org/10.1111/j.1749-6632.2009.04557.x

Poudrier, E., & Bell, B. J. (2025, August 13). Listener’s Perceived Emotions in Synthetic and Pitch-Randomized Music Excerpts. Retrieved from osf.io/vefy4

Poudrier, È., & Sapp, C. S. (2024, March). Suter (1980) Corpus. Polyrhythm, University of British Columbia (UBC). https://polyrhythm.humdrum.org

Poudrier, È., & Sapp, C. S. (2022). Polyrhythm analysis using the composite tool. In L. Pugin (Ed.), Proceedings of the 9th International Conference on Digital Libraries for Musicology (pp. 65–73). Association for Computing Machinery. https://doi.org/10.1145/3543882.3543890

Poudrier, È., & Shanahan, D. (2018). Modeling rhythmic complexity in a corpus of polyrhythm examples from Europe and America, 1900–1950. In R. Parncutt & S. Sattmann (Eds.), Proceedings of ICMPC15/ESCOM10 (pp. 355–360). Centre for Systematic Musicology, University of Graz. https://static.uni-graz.at/fileadmin/veranstaltungen/music-psychology-conference2018/documents/ICMPC15_ESCOM10%20Proceedings.pdf

Prince, J. B., Schmuckler, M. A., & Thompson, W. F. (2009). The effect of task and pitch structure on pitch-time interactions in music. Memory & Cognition, 37(3), 368–381. https://doi.org/10.3758/MC.37.3.368

R Core Team. (2021). R: A language and environment for statistical computing. In R Foundation for Statistical Computing [Online]. https://www.r-project.org

Repp, B. H. (1998). A microcosm of musical expression. I. Quantitative analysis of pianists’ timing in the initial measures of Chopin’s Etude in E major. Journal of the Acoustical Society of America 104(2), 1085–1100. https://doi.org/10.1121/1.423325

Robinson, D., Hayes, A., & Couch, S. (2022). broom: Convert Statistical Objects into Tidy Tibbles. In R package version 1.0.1 [Online]. https://CRAN.R-project.org/package=broom

Schellenberg, E. G., Krysciak, A. M., & Campbell, R. J. (2000). Perceiving emotion in melody: interactive effects of pitch and rhythm. Music Perception, 18(2), 151–171. https://doi.org/10.2307/40285907

Scherer, K. R. & Oshinsky, J. S. (1977). Cue utilization in emotion attribution from auditory stimuli. Motivation and Emotion, 1, 331–346. https://doi.org/10.1007/BF00992539

Scherer, K. R. & Zentner, M. R. (2001). Emotional effects of music: production rules. In P. J. Juslin and J. A. Sloboda (Eds), Music and Emotion: Theory and Research (pp. 361–392). Oxford University Press. https://doi.org/10.1093/oso/9780192631886.003.0016

Sears, D. R., Verbeten, J. E., & Percival, H. M. (2023). Does order matter? Harmonic priming effects for scrambled tonal chord sequences. Journal of Experimental Psychology: Human Perception and Performance, 4(7), 999–1015. https://doi.org/10.1037/xhp0001103

Senn, O., Kilchenmann L. , von Georgi R., & Bullerjahn C. (2016). The effect of expert performance microtiming on listeners’ experience of groove in swing or funk music. Frontiers in Psychology, 7, 1487. https://doi.org/10.3389/fpsyg.2016.01487

Steinbeis, N., Koelsch, S., & Sloboda, J. A. (2005). Emotional processing of harmonic expectancy violations. Annals of the New York Academy of Sciences, 1060(1), 457–461. https://doi.org/10.1196/annals.1360.055

Stupacher, J., Wrede, M., & Vuust, P. (2022). A brief and efficient stimulus set to create the inverted U-shaped relationship between rhythmic complexity and the sensation of groove. PLoS ONE, 17(5), e0266902. https://doi.org/10.1371/journal.pone.0266902

Turchet, L., Fazekas, G., Lagrange, M., Ghadikolaei, H. S., & Fishione, C. (2020). The internet of audio things: state-of-the-art, vision, and challenges. IEEE Internet of Things Journal, 7(10), 10233–10249. https://doi.org/10.1109/JIOT.2020.2997047

Wickham, H., Averick, M., Bryan, J., Chang, W., McGowan, L.D., François, R., Grolemund, G., Hayes, A., Henry, L., Hester, J., Kuhn, M., Pedersen, T.L., Miller, E., Bache, S.M., Müller, K., Ooms, J., Robinson, D., Seidel, D.P., Spinu, V., Takahashi, K., Vaughan, D., Wilke, C., Woo, K., Yutani, H. (2019). Welcome to the tidyverse. Journal of Open Source Software, 4(43), 1686. https://doi.org/10.21105/joss.01686

Witek, M. A., Clarke, E. F., Kringelbach, M. L., & Vuust, P. (2014). Effects of polyphonic context, instrumentation, and metrical location on syncopation in music. Music Perception, 32(2), 201–217. https://doi.org/10.1525/MP.2014.32.2.201