WHY is it that people find some musical or visual stimuli more appealing than others? In the decades-long search to understand the roots of such variation in "aesthetic pleasure" (Wassiliwizky & Menninghaus, 2021), several principles have been advanced to broadly account for people's responses. For instance, it has been variously proposed that individuals prefer stimuli to which they have been exposed more frequently (Zajonc, 1968), those that are processed more fluently (Reber et al., 1998), or those that are intermediate in complexity (Berlyne, 1971). Whereas considerable efforts have been devoted to adjudicating between these and other propositions (see Palmer et al., 2013, for a review), if there is one thing that the available empirical findings convergently demonstrate, it is that there are substantial individual differences in how distinct stimulus features influence aesthetic judgments. For instance, Hunter and Schellenberg (2011) found that frequency of exposure was often positively associated with liking for music among individuals lower in Openness-to-Experience, whereas it tended to be negatively associated with liking for music among those higher in the same trait. Likewise, in a series of studies, Güçlütürk and colleagues (2016; Güçlütürk & van Lier, 2019) showed that results ostensibly revealing an inverse curvilinear relationship between complexity and preference may actually reflect a mixture of two groups of individuals, one with a preference for simpler stimuli and another with a preference for more complex stimuli. A common theme among these various proposals is that preferences can be driven by abstract stimulus attributes (e.g., familiarity, fluency, complexity) that can, at least in principle, apply to any stimulus regardless of whether they are in the musical or visual modality.

In a recent article, Clemente and colleagues (2021) put forward particularly comprehensive and compelling evidence that aesthetic preferences among individuals are driven not by modality-general stimulus attributes, but by the specific ways in which different attributes manifest in different modalities. In their study, participants were asked to rate their liking for visual stimuli that were systematically developed to vary in their balance, whether their constituent elements (here, hexagons of different sizes) were spatially well-distributed versus clumped together; their contour, whether their constituent shapes featured smooth versus jagged vertices; their symmetry, whether they contained elements that mirrored one another across a dividing axis or featured different contents and/or arrangements of stimuli on different sides; and their complexity, whether they contained more or fewer constituent elements (here, geometrical shapes such as triangles or rectangles). Likewise, participants were asked to rate their liking for musical melodies that were designed to vary along analogous dimensions. Specifically, more balanced melodies featured a more homogeneous distribution of note onsets across the length of the musical line; "smoother" melodies featured smaller changes in pitch between adjacent notes and less abrupt rhythmic changes; more symmetrical melodies were more likely to involve the reversal of an initial sequence of notes (e.g., D-G-B-B-G-D) over the course of the melody; and simpler melodies contained fewer notes, with less variation in duration, pitch, and rhythm.

Using a novel analytic technique based on linear mixed modeling, Clemente et al. (2021) computed measures of how each participant's liking ratings were associated with changes in stimulus features along each dimension, what they labeled the individual's aesthetic sensitivity (AS) with respect to that dimension. Replicating their own earlier work (Corradi et al., 2020), Clemente et al. (2021) found that individuals showed considerable variability in their AS profiles. For instance, AS to visual symmetry was normally distributed with some participants showing a greater preference for symmetry and others for asymmetry. In addition, each visual feature—balance, contour, symmetry, and complexity—uniquely contributed to predicting individual liking ratings (see also Clemente, Friberg, & Holzapfel, 2023). Similar findings emerged for ratings of musical melodies, suggesting that participants also differed in terms of how each of the four stimulus dimensions affected their responses to the passages they heard (see also Clemente, Pearce, & Nadal, 2022). Together, these findings powerfully demonstrate the importance of taking AS into account in predicting and explaining aesthetic judgments of musical and visual stimuli, providing fertile ground for additional research, and potentially helping to account for past inconsistencies in the results of studies aimed at uncovering general principles of evaluative preference.

However, one of the most intriguing aspects of the work by Clemente et al. (2021) is what they did not find: Except for a single, moderate correlation between measures of AS to musical and visual contour, there were no reliable associations between stimulus features across sensory modalities. Based on this null finding, Clemente et al. (2021, p. 9) draw the provocative conclusion that "evaluative judgments of visual designs and melodies are not based on abstract representations of balance, symmetry, and complexity, but on visual- and auditory-specific instantiations of such attributes." Yet, upon consideration, the absence of a significant association between AS measures in different modalities is puzzling, particularly with respect to the complexity dimension. Theoretically speaking, it has been proposed that individuals differ quite broadly in their receptivity to objects and events that require more or less cognitive processing to comprehend (Cacioppo & Petty, 1982) or that involve more or less uncertainty (Webster & Kruglanski, 1994), suggesting that they should respond to variations in informational complexity in a manner that is consistent across stimulus domains and over time. In addition, research by both Palmer and Griscom (2013) and by Song et al. (2022) has revealed strong positive correlations between individuals' preferences for simplicity/harmoniousness versus complexity/inharmoniousness with respect to musical melodies/chords and combinations of colors. These empirical findings are difficult to square with the absence of any reliable cross-modal AS to stimulus complexity in the study by Clemente et al. (2021).

A close examination of the materials and measures used by Clemente et al. (2021) may help resolve this discrepancy. As alluded to by the authors themselves, the visual stimuli they used to test AS-to-complexity were static images made up of varying numbers and arrangements of geometric figures. However, melodies involve change over time, rendering the musical and visual stimuli in Clemente et al.'s study difficult to compare in terms of the amount of complexity they entail. Speculatively, this confound also opens the possibility that there may be distinct aesthetic sensitivities to stimuli that are static versus dynamic, a prospect that would be consistent with Clemente et al.'s overarching conclusion that individuals differ in their hedonic response to various attributes of a given stimulus.

It is also notable that Clemente et al. (2021) operationalized visual complexity via the number of elements contained in their static images. However, their operationalization of melodic complexity was itself more complex, involving a composite of event density and pitch-related entropy. Whereas the first component of the latter composite is akin to the number of elements in a visual figure in that it involves computing the number of notes per unit time, the second component captures the redundancy of the notes that appear within a given melody. This raises the possibility that a correlation between visual and musical AS might emerge if they were both operationalized in terms of entropy rather than number of elements. If so, it would be well in line with Nadal et al.'s (2010) contention that inconsistencies between studies testing the relationship between complexity and aesthetic judgments may result from differences in how complexity is defined and measured (see also Van Geert & Wagemans, 2020).

To assess these possibilities, we conceptually replicated part of Clemente et al.'s (2021) procedure for assessing cross-modal AS by administering the same melodic complexity measure that they used and collecting liking ratings for two new types of visual stimuli. The first type involved a display of colored lights in which the number of lights and timing of light onsets directly corresponded to the number and timing of notes in the melodies employed by Clemente et al. (2021). In this way, both the musical and visual stimuli were characterized by the same dynamics. The second type of visual stimulus involved texture patterns in which squares could be programmed to appear or not in each cell of a 10 x 10 grid. This enabled computation of the number of constituent elements (i.e., clusters of squares) within a particular texture pattern as well as the first order information entropy (i.e., binary symbol redundancy) of each pattern. As such, the complexity of these (static) visual stimuli could be operationalized in terms of the number and variety of elements in the figure, analogous to how complexity is operationalized in musical melodies. If there is evidence for a cross-modal AS-to-complexity association with the first set of stimuli alone, it would suggest that this correlation depends on stimuli in each modality having comparable dynamics. If there is instead evidence for cross-modal AS with both sets of stimuli, it would suggest that the correlation depends more broadly on measuring complexity in an analogous fashion for both visual and musical stimuli. Finally, if there is no evidence of a cross-modal AS-to-complexity association in either set of stimuli, it would conceptually replicate the null result reported by Clemente et al. (2021).

METHOD

Materials

MUSICAL STIMULI

Replicating the procedure of Clemente et al. (2021), we asked participants to rate the 24 melodies included in the MUST abridged Complexity subset (Clemente et al., 2020). These 4-second monophonic melodies in the key of C major vary in their complexity within the same loosely "classical" genre. The melodies were played using a synthesized piano timbre (WAV files of the melodies were downloaded from the MUST repository available at https://osf.io/bfxz7/).

VISUAL STIMULI: LIGHT DISPLAYS

To manipulate the modality of the aesthetic stimuli while controlling for their dynamic qualities as closely as possible, we created 24 light displays, each corresponding to one of the 24 MUST melodies described above. Specifically, each musical note appearing in the MUST melodies was mapped to an approximately 1 X 1 in. PNG graphic of a colored light bulb (see Figure 1). To account for octave equivalence—the tendency for listeners to perceive notes representing a doubling of frequency as part of the same category—notes sharing the same pitch chroma (e.g., C4 and C5) were mapped to images with the same hue. Light bulbs appeared on the computer screen in one of five rows, each of which represented a different octave, with lower rows representing higher pitch registers. Bulbs in each of seven columns shared the same pitch chroma (C, D, E, F, G, A, or B moving left to right along the Western C major scale). For instance, the bulb representing the note C3 appeared in the first column of the second row, whereas the bulb representing the note D5 appeared in the second column of the fourth row. The timing with which bulbs came on and off for a given light display was matched to the onset/offset time of notes in a particular MUST melody. However, given that the sequence of musical notes was quite rapid in some melodies, to prevent the corresponding bulb onset/offsets from blurring together, we reduced the tempo of the visual "melodies" by 50% such that each light display was stretched out to 8 s in length. This modification preserved the relative duration of each event in the sequence.

To ensure that participants with color vision deficiencies could adequately perceive the visual stimuli, musical notes were mapped to light bulb images using colors selected from the Okabe-Ito color-blind accessible palette available within the R programming environment (R Core Team, 2021). In addition, to ensure that our results were not dependent on a particular association between specific screen positions and specific colors, participants were randomly assigned to receive one of two pitch chroma to color mappings (see Table 1 for details). The first mapping was created arbitrarily by the authors and the second was created by randomly swapping colors originally assigned to appear on the left (e.g., D55E00) and right (e.g., CC79A7) sides of the screen. A video-recorded example of one of the light displays is available at: https://osf.io/34zqd/?view_only=e6f3ba01783643b2a4196ef2bda7bd82.

Figure 1. Colored light bulb images used to create light displays paired to each melodic stimulus.

VISUAL STIMULI: TEXTURE PATTERNS

To enable computation of measures of visual complexity based on both number of elements and their entropy, we presented participants with texture patterns modeled after those recently designed by Friedenberg and Liby (2016) to explore the association between visual complexity and aesthetic judgment. As alluded to above, each pattern involved presenting a set of gray squares within randomly selected cells of a 10 x 10 grid. Following Friedenberg and Liby, patterns were created within each of ten density levels from 10% to 100% in increments of 10% (see Figure 2 for examples). Participants were presented with five patterns within each of the first nine density levels and with two exemplars of a 100% density pattern, for a total of 47 images. Each pattern was presented against a black-colored background. Building upon Friedenberg and Liby's work, Gauvrit et al. (2017) developed algorithms for computing the number of distinct parts within each texture pattern as well as the first order entropy of each pattern (see Results for additional details).

Table 1. Pitch Chroma to Color Mappings

It should be noted that texture patterns with complementary density levels (e.g., 20% vs. 80%) shared the same entropy, avoiding any confound between entropy and the number of squares making up the pattern. Unlike the light display stimuli, these texture patterns were not systematically matched in their features with the melodies from the MUST set. However, this allowed us to assess whether AS-to-complexity would be positively correlated across modalities even when there was considerable variability in the specific, structural properties of the stimuli. As such, it enabled a stronger test of the role of modality-general representations of complexity in contributing to aesthetic preference.

Figure 2. Examples of texture patterns at three density levels.

RESULTS

DISCUSSION

It has long been recognized that people differ in how their preferences for musical sounds or visual images are influenced by stimulus complexity. However, the extent of overlap between individual differences in AS-to-complexity across the musical and visual domains has remained unclear. In their important recent study, Clemente et al. (2021) put forth evidence suggesting that aesthetic sensitivities to complexity in the musical and visual domains are uncorrelated. This lack of reliable cross-modal associations between AS measures led them to conclude that evaluative judgments of aesthetic stimuli are not based on modality-general representations of complexity. However, we hypothesized that the inability of Clemente et al. (2021) to detect cross-modal associations between AS-to-complexity for musical versus visual stimuli may have at least partly resulted from a confound between modality and temporal dynamics in their experimental stimuli and/or a confound in their operationalization of complexity for stimuli in different modalities.

To test these possibilities, we conceptually replicated part of their procedure using visual images and musical melodies that were either very closely matched in their temporal dynamics and/or for which complexity was operationally defined in a very similar way (either in terms of number of elements or their information entropy). Our results revealed that there was indeed a reliable overall association between AS-to-complexity within the musical and visual domains, even when the stimuli differed in their temporal dynamics, so long as complexity was measured using similar metrics that account for entropy. This suggests that Clemente et al. (2021) underestimated the role of modality-general representations of complexity in evaluative judgment. More broadly, our results serve to reconcile Clemente et al.'s (2021) findings regarding AS-to-complexity with theories that posit modality-general individual differences in preference for information differing in complexity (e.g., Cacioppo & Petty, 1982), with theories of aesthetic preference that are based on domain-general cognitive mechanisms of prediction and processing fluency (e.g., Brielmann & Dayan, 2022), and with other recent empirical studies that appear to show quite robust cross-modal commonalities in preferences for complexity (e.g., Palmer & Griscom, 2013; Song et al., 2022).

It is important to note that the present findings are silent regarding why Clemente et al. (2021) did not show cross-modal aesthetic sensitivities for stimulus attributes other than complexity. As noted earlier, Clemente and colleagues found that AS to visual balance and symmetry were not reliably associated with AS to musical balance and symmetry. Although these null findings may point to the involvement of modality-specific representations of sensory information for the same attributes, it is also possible that the attributes described by the terms "balance" and "symmetry" are qualitatively different between visual and musical modalities. In other words, the senses of the terms "balance" and "symmetry" when used to describe the distribution of visual elements within a two-dimensional frame may be largely unrelated to the senses of those same terms when used to describe the distribution and ordering of musical notes in time. For instance, it is unclear whether individuals construe melodies in terms of isochronal sections, akin to spatial "sides", over which they might cognitively represent the extent to which the melodic sequence in the first section has been reversed in the second. Detecting cross-modal commonalities in aesthetic preference may require ensuring that the materials and measures implemented to assess AS within each modality are closely aligned, as we did in the present work concerning "complexity". Future work may thus focus on how terms like "balance" and "symmetry" are applied to different types of materials, enabling the development of formal measures of these constructs that can be applied to stimuli from multiple modalities as we did when using entropy to operationalize "complexity".

Although our results appear to be inconsistent with one of the conclusions drawn by Clemente et al. (2021), they resoundingly support another of their primary contentions: We find evidence of substantial variability in individuals' aesthetic sensitivities to complexity. Histograms of AS for each of the three stimulus types in our study are shown in Figure 3. These show that scores on all three measures are roughly normally distributed with a considerable proportion of participants showing positive responses to increased complexity, others showing negative responses, and a great many showing relative indifference to this attribute (cf. Güçlütürk et al., 2016; Güçlütürk & van Lier, 2019). Likewise, it appears that even when stimuli in different modalities are matched very closely in terms of their objective complexity, individuals markedly vary in terms of the extent to which they assign these stimuli similar ratings. For instance, whereas the average correlation between liking ratings for melodies and matched light displays was significantly positive in the present study, correlations at the individual level ranged from -.58 to .90. Considering such variability, it is possible that the conclusions of Clemente et al. (2021) were subject to sampling variability that we were able to characterize more fully using our comparatively larger sample. In addition, the modest overall correlations we report leave considerable room for modality-specific mechanisms to operate. However, the presence of only modest modality-general effects does not imply that the remaining variance in evaluative judgments must be due to modality-specific effects. As noted earlier, within-modality correlations in our study were also rather modest, opening the possibility that modality per se may not be as essential to evaluative judgments as Clemente et al. (2021) have argued.

Figure 3. Frequency distributions of AS-to-Complexity scores for (A) light displays; (B) melodies; and, (C) texture patterns. Vertical dashed lines represent insensitivity (i.e., indifference) toward a given aesthetic attribute.

In sum, the findings of our conceptual replication study stand to help resolve a theoretical and empirical conundrum posed by the intriguing results of Clemente et al. (2021), while validating their overall methodological approach to elucidating individual differences in AS across modalities (see also Clemente et al., 2020; Clemente, Pearce, & Nadal, 2022; Corradi et al., 2020). The study also introduces a novel paradigm for equating the temporal dynamics of musical and visual stimuli—involving light displays that constitute a form of "visual music"—and newly demonstrates the utility of random texture patterns (Friedenberg & Liby, 2016; Gauvrit et al., 2017) in equating measures of different forms of complexity across modalities. As such, we hope that the work will be of value to scholars of music and aesthetics, whose daunting challenge to account for evaluative preferences is laid bare by the provocative recent contributions of Clemente and colleagues.