How Music Moves

04.MUSIC.23_221-248.qxd 01/02/2006 12:26 Page 221 How Music Moves: Musical Parameters and Listeners’ Images of Motion 221 H OW M USIC M OVES : Musical Parameters and Listeners’ Images of Motion R Z OHAR E ITAN Tel Aviv University ECENT YEARS HAVE REVEALED increasing interest by R ONI Y. G RANOT The Hebrew University of Jerusalem THIS ARTICLE PRESENTS AN empirical investigation of the ways listeners associate changes in musical parameters with physical space and bodily motion.1 In the experiments reported, participants were asked to associate melodic stimuli with imagined motions of a human character and to specify the type, direction, and pacechange of these motions, as well as the forces affecting them. The stimuli consisted of pairs of brief figures, one member of a pair presenting an “intensification” in a specific musical parameter, the other an “abatement” (e.g., crescendo vs. diminuendo, accelerando vs. ritardando). Musical parameters manipulated included dynamics, pitch contour, pitch intervals, attack rate, and articulation. Results indicate that most musical parameters significantly affect several dimensions of motion imagery. For instance, pitch contour affected imagined motion along all three spatial axes (not only verticality), as well as velocity and “energy.” A surprising finding of this study is that musical-spatial analogies are often asymmetrical, as a musical change in one direction evokes a significantly stronger spatial analogy than its opposite. Such asymmetries include even the entrenched association of pitch change and spatial verticality, which applies mostly to pitch falls, but only weakly to rises. In general, musical abatements are strongly associated with spatial descents, while musical intensifications are generally associated with increasing speed rather than ascent. The implications of these results for notions of perceived musical space and for accounts of expressive musical gesture are discussed. Received February 13, 2004, accepted June 30, 2004 1 The title “How Music Moves” is derived from Kivy (1990, chapter 8). Music Perception VOLUME 23, ISSUE 3, PP. 221-247, ISSN diverse scholarly domains in the role of crossmodal interactions in music, and particularly in the associations of music with physical space and bodily motion. Music theorists influenced by Lakoff and Johnson’s notions of embodied metaphor and image schemas (Johnson, 1987; Johnson & Larson, 2003; Lakoff & Johnson, 1980) have used cross-domain mapping as a basis for musical analysis, theoretical models of musical structure, and meta-theoretical accounts of musical discourse (Brower, 2001; Larson, 1997; Saslaw, 1996; Spitzer, 2004; Zbikowski, 1997, 2002). Analogies of musical figures and bodily gestures provide for the “iconic” component in the Piercean notion of musical signification, as interpreted by recent theories of musical semiotics (Cumming, 1997; Hatten, 1997-2002, 2005; Lidov, 1987, 1999). Concurrently, theoretical and empirical studies are investigating the interaction of music with visual and kinesthetic stimuli in the perception of film, dance, and other forms of “musical multimedia” (Bolivar, Cohen, & Fentress, 1994; Cook, 1998; Krumhansl & Schenck, 1997; Lipscomb, 2005; Lipscomb & Kendall, 1994). Other works investigate empirically the role of performers’ movements and gestures in conveying structural and expressive information (Ashley, 2003; Clarke & Davidson, 1998; Davidson, 2002; Iyer, 2002; Vines et al., 2004; Williamon & Davidson, 2002), and applied research produces sophisticated digital devices mapping bodily motion and gesture into auditory information (Camurri & Volpe, 2003; Wanderley & Battier, 2000). Indeed, perceptual experiments, using simple auditory stimuli, have suggested that auditory parameters such as pitch height or loudness and visuospatial features such as size, shape, and height interact in perception (see Marks, 2000, for a survey of the literature). Kubovy & Van Valkenburg (2001), following Angell (1906), propose that an auditory “where” system, serving visual orientation, may account, at least partially, for such interactions between the auditory and visuokinetic modalities. “Auditory localization occurs in the space world of vision-touch-movement . . . Most persons seem to make their localization of sounds either in the form of visual imagery, or in the form of quasireflex localizing movements of the head and eye” 0730-7829, ELECTRONIC ISSN 1533-8312 © 2006 BY THE REGENTS OF THE UNIVERSITY OF CALIFORNIA . ALL RIGHTS RESERVED. PLEASE DIRECT ALL REQUESTS FOR PERMISSION TO PHOTOCOPY OR REPRODUCE ARTICLE CONTENT THROUGH THE UNIVERSITY OF CALIFORNIA PRESS ’ S RIGHTS AND PERMISSIONS WEBSITE AT WWW. UCPRESS . EDU / JOURNALS / RIGHTS . HTM 04.MUSIC.23_221-248.qxd 222 01/02/2006 12:26 Page 222 Z. Eitan and R. Y. Granot (Angell, 1906, pp. 154-155, quoted in Kubovy & Van Valkenburg, p. 100). Sound would thus almost inevitably activate visual and kinetic imagery. Support for such notions of cross-modal perception is supplied by converging neurological evidence, suggesting that specific subcortical areas such as the superior colliculus combine visual, auditory, and somatosensory information into an amodal spatial representation (e.g., Spence & Driver, 1997; Stein, Wallace, & Meredith, 1995). Further evidence for the relationship between sound and visuokinetic imagery is provided by studies reporting the activation of brain areas generally associated with visuospatial processing during music-related tasks (Nakamura et al., 1999; Penhune et al., 1998; Platel et al., 1997; Zatorre, Evans, & Meyer, 1994). Musical–Motional Analogy Of the ideas associating music with nonauditory domains, the notion that music depicts analogues of physical, particularly human, motion is probably the oldest and the most influential.2 Aestheticians have used this analogy to account for (or sometimes, as with Hanslick, to discount) musical affect, suggesting that musical structures evoke emotion through isomorphism with expressive human motion (Cook, 1998; Davies, 1994; Hanslick, 1891/1986; Kivy, 1980; Langer, 1953; Scruton, 1997). The idea that musical gestures and phrases are analogous to natural motion is also central in “energetic” theories and analyses (conceiving music as generated by and expressive of the flux of mental tension and relaxation) suggested by a number of music theorists, most notably Ernst Kurth (1917/1971, 1931/1969, 1991). This idea also generated attempts to map changes in musical parameters into curves analogous to motion contours, used as analytic tools and as guidelines for performers (see Repp, 1993, and Shove & Repp, 1995, on early-twentieth-century theorists Becking, Sievers, and Truslit; for a recent example, see Rink, 1999). Independently of the work of music theorists, psychologists of music have tried in recent decades to examine analogies of music and motion empirically. The bulk of this research (briefly surveyed below) centers upon music performance. Less attention has been given to the examination of the analogy of music and motion from a listener’s perspective. As Clarke (2001) points out, different, though possibly interlinked, 2 For an early version of this notion, see Aristoxenus of Tarentum (third century A.C.) in Barker (1989, p. 186). See also Cohen (2001) for a discussion of the Aristotelian sources of the Western concept of music as directed motion. sources may contribute to a listener’s experience of music as motion: the motions of an actual or imagined performer; a sense of self-motion induced by the music’s expressive properties (as Todd, 1992, proposed); or the experience of music in terms of metaphorical motion in virtual space, supported by mappings such as that of pitch and verticality (see Gjerdingen, 1994, for an account of such experiences as analogous to apparent motion effects in vision). The present article aims at investigating empirically such “metaphorical” experience of music as spatial motion, often considered pertinent to musical experience (Kurth, 1991; Scruton, 1997). The initial hypothesis of this article echoes Clarke’s (2001) suggestion that “since sounds in the everyday world specify (among other things) the motional characteristics of their sources, it is inevitable that musical sounds will also specify the fictional movements and gestures of the virtual environment which they conjure up.” Stated in empirically testable terms, this hypothesis implies that changes in specific musical parameters (such as pitch height, loudness, or attack rate) would be associated by listeners with corresponding changes in specific dimensions of motion (such as spatial verticality, distance, or speed). Here, we examine this hypothesis by investigating how music affects listeners’ mental images of bodily motion; in particular, we examine whether changes in musical parameters evoke corresponding changes in listeners’ spatial and kinetic imagery.3 Mapping Music Into Space and Motion: Specific Parameters Some analogies between specific musical and motional parameters may easily suggest themselves: temporal features, such as tempo or attack rate, are associated with speed or velocity; changes in pitch, with spatial ascent and descent, as well as distance change; loudness, with both distance (approaching or moving away from the listener) and the level of energy which activates motion. The following is a brief survey of the empirical evidence suggesting that such analogies take part in listeners’ experience of music. Perhaps the most apparent of all musical–motional relationships is the association between tempo (or, 3 Note that we do not focus here on the most basic and primary relationship between music and motion, evident in dance and other music-related organized movements (e.g., Wallin, Merker, & Brown, 2000), but rather on a more abstract type of relationship, which emerges when music is disconnected from its bodily roots, as has occurred in Western music. Nonetheless, it may well be that these abstract relationships are an expression of the embodied nature of music, which sets it aside from other arts such as painting or poetry. 04.MUSIC.23_221-248.qxd 01/02/2006 12:26 Page 223 How Music Moves: Musical Parameters and Listeners’ Images of Motion when a steady pulse is absent or ambiguous, attack rate) and the speed of human motion. Coordination of the pace of motor behavior with perceived auditory patterns begins in infancy (Malloch, 2000; Maurer, 1993; Papousek, 1996; Stern, 1985; Sullivan & Horowitz, 1983) and progresses throughout early childhood (Moog, 1976). The pace of motion is universally coordinated by musical tempo in dances, marches, and other motion-related musical genres. Several attempts have been made to model tempo variations in performed music in terms of physical motion (Feldman, Epstein, & Richards, 1992; Repp, 1992a; Todd, 1992). More concretely, expressive tempo variations were modeled on the speed or velocity profiles of human locomotion, comparing, for instance, the final ritardando (the slowing down that characterizes the performance of section endings) with runners’ decelerations (e.g., Friberg & Sundberg, 1999; Kronman & Sundberg, 1987; Sundberg & Verrillo, 1980). Empirical corroboration for such models comes from performance data (average tempo profiles) and from comparisons of listeners’ preferences for different tempo curves. Several studies suggest that correspondence of expressive timing and bodily motion affects listeners, as well as performers. Repp (1992b) shows that listeners prefer parabolic timing curves of tempo change over other functions of tempo change. Juslin, Friberg, & Bresin (2002) report that listeners rate tempo curves derived from human motion profiles as more musical and expressive than simple tempo changes. Repp (1998) demonstrates that trained listeners’ ability to detect a specific inter-onset interval (IOI) lengthening in a generally isochronous musical phrase is related to the relative lengthening of the relevant note in a typical expressive performance of the phrase. Repp relates this finding to kinematic implications of musical structure, which induce perceptual biases in listeners. The validity of biological motion models to musical rhythm and tempo has, however, been contested. Honing (2003; see also Desain & Honing, 1993) points out that these models are insensitive to note density, rhythmic structure, and global tempo and fail to specify how the metaphorical parameters of mass and speed, necessary to resolve the mechanical equations, can be derived from the music.4 4 The structure of rhythmic patterns (as distinguished from tempo or expressive timing) may also be associated with the perceived features of physical motion, as shown by Gabrielsson (1973a, 1973b). Gabrielsson points out that “the relations between ‘rhythm’ and various aspects of ‘movement’ are notoriously hard to analyze” (1973a, p. 259). Still, he proposes, based on a factor analysis of listeners’ adjective ratings of rhythmic figures, a number of dimensions that reflect experienced “movement characters” such as floating-stuttering, dancing-walking, or solemn-swinging (1973a). 223 While the relation between tempo and motion may be self-evident, the relationship between pitch and verticality is less straightforward. Western musicians and listeners habitually speak of pitch in terms of vertical positions and motions (high and low, rise and fall). Yet the sources of such verticality metaphors and their role in the listener’s musical experience are in dispute. The influence of Western musical notation is often suggested as the prime factor in shaping the concept of pitch verticality. Accordingly, this relationship would be culturally and historically rooted, rather than universal or biological in origin. Indeed, as Zbikowski (1998) points out, alternative characterizations of pitch positions have been used: “sharpness” and “heaviness” in ancient Greece, “small and “large” in Bali and Java, or “young” and “old” among the Suyá of the Amazon basin. Still, some hypotheses propose an embodied relationship of pitch and vertical space. Cox (1999), for instance, following Lakoff and Johnson (1980), relates pitch verticality to the general metaphorical mapping “greater is higher” (e.g., rising violence, higher salaries) via the experience of vocal production, in which greater quantities or magnitudes of air, effort, and tension indeed produce higher notes. Debates notwithstanding, empirical works do suggest that pitch height is strongly associated, at least by Western subjects, with spatial verticality. This association, applying to both musicians and nonmusicians, affects not only conscious cognitive processes but also unconscious perceptual ones. Thus, the perceived spatial elevation of a sound increases as its pitch increases (Pratt, 1930; Timble, 1934; Roffler & Butler, 1968). In speeded discrimination tasks, congruence of spatial and pitch position positively affects response speed both when subjects discriminate high and low pitches and (though to a lesser degree) when discriminating high and low positions in space (Bernstein & Edelstein, 1971; Melara & O’Brien, 1987). In a recent experiment, Widmann et al. (2004) demonstrated that infringement of the spatial associations of pitch elicits an almost immediate brain response: when an auditory stimulus was incongruent with visual information (e.g., when a higher pitch corresponded to a lower visual signal), brain responses starting as early as about 100 ms from the onset of the auditory stimulus were detected. Note that while the above studies refer to the association between relative pitch height and spatial location, studies indicate mapping between motion on the vertical axes of pitch and space as well (Walker, 1987; Lipscomb & Kim, 2004). Results are conflicting, however, with regard to the nature of the pitch-verticality association. Some studies 04.MUSIC.23_221-248.qxd 224 01/02/2006 12:26 Page 224 Z. Eitan and R. Y. Granot suggest an innate or at least easily learned perceptual relationship. Thus, Wagner et al. (1981), using a selective-looking paradigm, found that 1-year-old infants match tones that rise or fall in sound frequency with arrows that point up or down in space, respectively; Roffler and Butler (1968) report that pitch height affects perceived spatial elevation in congenitally blind participants and in children 4 to 5 years old who did not associate pitch with height verbally. Other studies, however, indicate a learned response, probably related to linguistic and notational convention. Ashley (2005), who examined how visual contours presented simultaneously with a melody affect musically trained listeners’ memory for the melody, found that correspondence of visual and melodic contour facilitated performance, as compared to matching a melody with an inverse visual contour. However, differences between the two conditions were obliterated following some training, suggesting that pitch verticality mappings are learned rather than innate. Indeed, preschool children, though able to discriminate pitch registers, rarely use the terms “high” and “low” to describe pitch (Hair, 1981). Moreover, the cross-modal reference of the terms “high” or “low” seems to impair English-speaking children’s ability to describe pitch verbally, as compared to children who speak French and Spanish, where specific rather than cross-domain terms (“aigu” and “grave” in French; “agudo” and “grave” in Spanish) describe pitch register (Abril, 2001; Costa-Giomi & Descombes, 1996; Flowers & Costa-Giomi, 1991). More surprisingly, auditory pitch has also been associated with lateral position, so that higher pitch was found to be related to right-side position and lower pitch to left-side position. Several research paradigms have indicated this relationship. Mudd (1963) asked subjects to listen to pairs of sounds and represent the two-dimensional spatial position associated with these sounds by placing pegs on a pegboard. Spatial position and pitch were associated both vertically and laterally, as higher pitches were generally positioned above and to the right of lower pitches. More recently, Stevens and Arieh (2005) asked subjects to detect rapidly a visual target in one of two horizontally aligned boxes on the computer screen, while ignoring a tone (high or low) preceding the target by 100–500 ms. They found that reaction time was significantly faster when pitch and lateral position were congruent (high-right, low-left), though only when the tone preceded the visual target by 400 ms or more. Wühr & Müsseler (2002) used a dualtask paradigm, in which a speeded left or right response to a low or high tone was combined with the identification of a masked left- or right-pointing arrow following the tone. Pitch (high or low) interfered with the lateral identification task. In contrast with the above experiments, which suggest a general pitch-laterality association, Stewart, Walsh, and Frith (2004), using a stimulus-response compatibility task, found a right-up left-down advantage only for pianists. They suggest that results are due to visuomotor mapping from the vertical layout of pitch notation to its horizontal layout on the keyboard, learned through piano playing and generalized to nonmusical tasks. Another relationship, well entrenched in our daily experiences, is that between distance and loudness. In general, loudness decreases as a function of the change in distance raised to the power of two. In music, dynamic changes are mostly produced by changes in the energy of the emitted sound rather than changes in the distance of the perceiver from the sound source. Nonetheless, a listener might metaphorically relate musical loudness to distance, given a lifelong experience of relating the two features in nonmusical contexts. A change in distance between the sound source and the perceiver may also influence the perceived pitch, as demonstrated by the Doppler effect (Neuhoff & McBeath, 1996). If this effect is generalized by listeners, they may tend to associate rise in pitch with decrease in distance, and pitch fall with increase in distance. Crossovers: Relating Music and Motion Through Intensity Isomorphism Beyond associations of motional gestures with specific parameters, musical and motional gestures may also be related by way of analogous intensity contours. We may conceive of changes in many different domains as “increases” or “decreases” in a given parameter. In music, for instance, loudness increases and decreases, pitch “rises” and “falls,” and attack rate accelerates (increases pace) or decelerates. Likewise, bodily motion may ascend (increase elevation, as well as effort and tension) or descend, speed up (increase pace) or slow down. One common cross-domain aspect of such processes may be described as changes in the level of intensity (intensifications vs. abatements). A pitch rise, a crescendo, and an accelerando are commonly considered intensifying (at least by musicians), as are speeding up or ascending in the domain of human motion. These crossparametric influences should also be taken into consideration when interpreting the data relating specific musical parameters and motional gestures. A summary of this topic can be found in Eitan and Granot (2005a). Converging evidence from a variety of disciplines suggests that comparable intensity levels or intensity 04.MUSIC.23_221-248.qxd 01/02/2006 12:26 Page 225 How Music Moves: Musical Parameters and Listeners’ Images of Motion contours may indeed associate different perceptual and conceptual domains. Various research paradigms provide empirical support for a cross-dimensional notion of intensity, including cross-dimensional matching tasks (Stevens, 1961, 1975; Stevens & Guirao, 1963), crossdimensional interference in speeded classification (Melara & Marks, 1990a, 1990b, 1990c; see Marks, 2000, for a survey of other works), and cross-dimensional effects on perceived intensity (see below). Additional corroboration for both cross-modal and intra-modal intensity analogies is supplied by studies of adult-infant communication (Malloch, 2000; Maurer, 1993; Papousek, 1996; Stern, 1985; Sullivan & Horowitz, 1983). These indicate that infants and parents often connect through mutual imitation of cross-modal intensity contours, involving auditory dimensions such as pitch inflection and dynamics, as well as touch and motion. Also relevant are “sonification” experiments, relating auditory intensity to diverse conceptual domains. For instance, Walker (2000) successfully used pitch and tempo contours as icons for temperature, pressure, velocity, size, and quantity changes. Together, these studies suggest that intensity associates auditory and nonauditory domains (e.g., pitch and brightness, loudness and size; see Marks, 2000). Of special relevance are, in addition, studies suggesting that intensity changes in one auditory dimension may create an illusion of corresponding changes in another. Such intra-modal relationships may produce “crossing over” of the kinetic associations of one musical parameter (e.g., pitch and verticality) to another (e.g., dynamics). Neuhoff et al. (Neuhoff & McBeath, 1996; Neuhoff, McBeath, & Wanzie, 1999) found that changes in loudness (crescendo and diminuendo) create an illusion of congruent pitch changes (rise and fall), and vice versa (see also Nakamura, 1987). Tekman (1997) reported that pitch accents sound louder, and IOI preceding dynamic (loudness) accents sound longer; pitch perception, however, was not affected by dynamic accents. In another study, Collier and Hubbard (2001) found an interaction between pitch and tempo: accelerations were perceived as faster in higher pitches, and decelerations were perceived as slower in lower pitches. Interactions between pitch and loudness as well as with timbre were also found by Marks and Melara (Marks, 1987; Melara & Marks, 1990a, 1990b). Eitan and Granot (2005b) found that listeners assess stimuli with isochronous IOI as accelerating if loudness or textural density are progressively increased, and as decelerating if decreased. Likewise, isomorphic intensity contours in different parameters (e.g., an accelerando, a crescendo, and a pitch rise) significantly affected similarity ratings 225 of musical figures (Eitan & Granot, 2003, 2005a). Thus, the musical dimensions of loudness, pitch, and tempo seem to interact via concomitant intensity levels or contours. Intensity Contours in Music The notion of an intensity or tension “curve” that plots the dynamic progression of a musical segment or composition, determined by the combined ebb and flow in various musical dimensions, has been independently suggested by several music theorists throughout the twentieth century, most notably Kurth (1917/1971, 1931/1969, 1991). Such contours were often proposed as analogues of either metaphorical, psychological “motion” or actual physical movement. Thus, for Kurth, the actual acoustical manifestation of music (Erscheinungsform) is generated by a primordial “kinetic energy” (Bewegungsenergie), the basis for musical continuity, and by a “play of [psychological] tensions” (Spiel von Spannungen), producing curves of musical intensification and abatement. His basic unit of musical growth and decay, the “linear phase,” is thus not defined by specific pitches, intervals, or durational patterns but by its global growth contour (1917/1971, p. 21ff). In his musical analyses (e.g., of Bruckner’s symphonies; see Kurth, 1991, pt. III), Kurth often examines overall intensifying and abating processes, generated by the combined activity of diverse parameters, including rhythmic and textural density, contour and register, timbre, and tonal distance. These create intensity “waves” of different scales, whose interaction is the basis for a piece’s motional shape. Conceptual frameworks similar to Kurth’s were suggested, more recently, by Agawu (1982), Berry (1976), Meyer (1989), and Rink (1999). Diverse concrete mappings of the contours of musical intensity into physical motion have been proposed by early-twentieth-century musicians (e.g., Truslit, Becking, Sievers; see Shove & Repp, 1995), movement theorists (e.g., Jaques-Dalcroze, 1921/1967), and film theorists and practitioners, such as Eisenstein (see Cook, 1998). While discussing this extensive body of work is beyond the scope of this article, a few recent empirically oriented models of intensity contours in music should be mentioned. Todd (1992, 1994, 1995, 1999) has proposed a model relating intensity contours in music to human motion perception. He suggests that two sensory mechanisms, the vestibular and the audiovisuo-motor, translate auditory stimuli into kinetic information. This translation, which applies to two kinds of motion, gestural (continuous expressive 04.MUSIC.23_221-248.qxd 226 01/02/2006 12:26 Page 226 Z. Eitan and R. Y. Granot motion) and locomotive (associated with tempo in metrical music), is mediated through the integrated intensity profile generated by various psychoacoustic parameters (tempo, dynamics, articulation, timbre, and vibrato) throughout a piece or section. The proposed relationship between intensity curves and motional gestures is echoed in Manfred Clynes’ notion of “essentic forms,” assumed to reflect archetypical icons of basic human emotions (Clynes & Nettheim, 1982). These contours may be depicted aurally through pitch and loudness, as well as through other modes, such as touch, motion, and visual curvature. Clynes demonstrated that the emotions associated with essentic shapes may be correctly identified cross-culturally when these shapes are presented through sound, vision, or haptic pressure. Regardless of his theory of “sentics” (which has not been widely accepted), Clynes’ results reveal the ability to associate intensity contours crossmodally. The cross-modal relationship of motion, emotion, and musical features is exemplified by a number of more recent studies which demonstrated that the perceived “musical tension,” as measured by the Continuous Response Dial Interface (e.g., Fredrickson, 1997; Krumhansl, 1996; Madsen & Fredrickson, 1993; Nielsen, 1983), seems to correlate with the tension conveyed by human motion. Indeed, in studies in which listeners continuously rated the degree of musical tension in a musical piece, Krumhansl and Schenck (1997), as well as Frego (1996), demonstrated that participants who listened to a piece of music and spectators of a choreography composed to that music concurred in their tension ratings, even though the listeners have not watched the dance and spectators have not listened to the music. These results suggest that the tension fluctuations conveyed by motion parameters correlate with those associated with musical parameters. Hypotheses The experiments reported here aim to investigate how listeners associate changes in various musical parameters with aspects of motion in space. Specifically, we examine how Western listeners associate simple and controlled musical stimuli (associated with Western musical tradition) with images of motion in space. These experiments are a part of an ongoing series, utilizing a variety of tasks and musical stimuli (Eitan & Granot, 2005c, report on further experiments). Based on studies reviewed in the introduction, we set out to test the following specific hypotheses. Associations of Specific Parameters As suggested above, we may hypothesize that participants will associate • IOI changes and speed changes • Changes in pitch with changes in spatial verticality • Pitch changes with changes in distance (pitch rise would imply approaching motion, and pitch fall would imply motion away) • Loudness changes with changes in distance • Loudness changes with changes in “energy” Intensity “Crossovers” Music and motion may be related through crossdimensional intensity both directly and indirectly. Directly, similarity of intensity contours may affect the mapping of music into motion: musical processes perceived as intensifying would map into intensifying motions, and musical abatements would map into abating motions. Indirectly, an intensity contour in one musical dimension may be associated with that of another (e.g., a diminuendo and a pitch fall) and thus draw on the latter’s kinetic associations (e.g., a diminuendo might be perceived as “falling”). With regard to the present study, such relationships would suggest the following hypotheses: 1. Correlation of intensity vectors in music and motion. Intensifying changes in musical stimuli would evoke corresponding images of kinetic intensifications, while musical abatements would be associated with motions implying decreasing intensity. A subsidiary hypothesis, following from that above, is: 2. One-to-many mapping. Each musical parameter may map onto several motional features presenting analogous intensity profiles. For instance, a pitch rise may evoke, in addition to spatial ascent, a sense of acceleration and approaching movement. Correspondingly, each kinetic feature may be evoked by several analogous musical parameters. For instance, a spatial ascent may be evoked by pitch rise, crescendo, and accelerando. We also hypothesize that associations between music and motion are directionally symmetrical: 3. Symmetry of associative space. Other things being equal, diametrically opposed musical processes ⬍m, 04.MUSIC.23_221-248.qxd 01/02/2006 12:26 Page 227 How Music Moves: Musical Parameters and Listeners’ Images of Motion –m⬎ would evoke diametrically opposed kinetic processes ⬍k, –k⬎. In experimental terms: a listener who associates a musical stimulus m (e.g., a crescendo) with a kinetic quality k (e.g., a spatial ascent) would associate the inverse stimulus –m (e.g., diminuendo) with the opposite kinetic quality –k (e.g., descent). The Effect of Overall Tempo Pace is the most direct link between music and motion and has been shown to strongly affect both music perception and production. For instance, similar rhythmic figures are perceived and performed differently in different tempi (Repp, Windsor, & Desain, 2002; Windsor, Aarts, Desain, Heijink, & Timmers, 2001). It is thus important to investigate how an overall change in tempo affects the spatio-kinetic associations of dynamic changes in musical parameters. To examine this issue, we performed two versions of our experiment, using the same musical stimuli in two different tempi (see Experiments 1 & 2 below). This investigation could also contribute to the examination of a more general issue: namely, how does an overall change in the range of an auditory parameter affect the perception of dynamic auditory stimuli? (For relevant studies, see Collier & Hubbard, 2001; Neuhoff, McBeath, & Wanzie, 1999). Methods Experiment 1 PARTICIPANTS Seventy-eight Tel Aviv University students (mean age ⫽ 24.9, SD ⫽ 7.9; 45 females, 33 males) participated in the experiment. Thirty-seven of these (20 females, 17 males) had at least 7 years of music training (“musicians”), while the remaining 41 (25 females, 16 males) had little or no formal music training. TASK AND PROCEDURE Participants were asked to visualize an animated (“cartoon”) human character of their choice.5 They then heard brief melodic figures, and for each figure had to visualize their character moving in an imaginary animated film shot, the melody serving as its “soundtrack.” For each melodic figure, participants specified their 5 In other experiments, reported elsewhere (Eitan & Granot, 2005c), we employ tasks involving imagining self-motion, rather than an external cartoon character. See also p. 241. 227 character’s imagined motion in a forced-choice questionnaire as follows: 1. Motion type: specified categories included walking, running, jumping, crawling, and falling/sliding (however, participants could mark and describe any other motion, not only locomotion types) 2. Vertical direction: ascending, descending, or level 3. Change of (virtual) distance from spectator: approaching, moving away, or neither 4. Direction on the horizontal plane: motion to the right (relative to the spectator), left, or neither 5. Change of the character’s pace 6. Whether an external force (excluding gravity) interferes with the imagined motion 7. Whether this force supports, opposes, or sidetracks the imagined motion. 8. The character’s “energy” level for each motion (on a 1-7 scale) In addition, participants could write a brief free description of the imagined motion for each stimulus. Note that though most forced-choice questions may be primarily associated with locomotion, participants were free to imagine any type of motion. When choosing types of motion not involving change of spatial location, they were asked to select the entry “neither” (in questions 2-4) and to add comments on the response sheet. After instructions were read to the participants and each participant had chosen an imaginary character, participants heard each stimulus three times in succession, with approximately 5 seconds of pause between presentations. Stimuli were then presented again after a 1-minute interval. Subjects were given approximately 3 minutes to complete the task for each stimulus. Questionnaires were presented in Hebrew, the participants’ native language. Stimuli were presented in monophonic recording through two loudspeakers. Participants were seated at a roughly equal distance from both loudspeakers. They heard the stimuli in groups of 5-8 people, in a single session of 45-50 minutes. Stimuli were presented to each group in a different random ordering (altogether, 11 randomizations were produced). Opposite stimuli (e.g., stimuli 1 & 2) were not presented in immediate adjacency. EXPERIMENTAL MATERIALS The musical stimuli consisted of pairs of brief melodic figures. One member of each pair presented an “increase” (intensification) in a specific musical parameter, while the other presented a “decrease” (abatement)—for 04.MUSIC.23_221-248.qxd 228 01/02/2006 12:26 Page 228 Z. Eitan and R. Y. Granot instance, crescendo versus diminuendo, or accelerando versus ritardando. Other parameters were held constant for each pair. Parameters investigated were dynamics, pitch contour (ascending vs. descending), pitch interval size, inter-onset intervals (IOI or attack rate), motivic pace (the duration of successive melodic figures), and articulation (staccato vs. tenuto). To minimize the effects of tonal implications, all stimuli were tonally ambiguous. Figure 1 presents the stimuli used (BPM ⫽ 160). Stimuli 1 and 2 are a crescendo and diminuendo over a repeated pitch. Stimuli 3 and 4 are direct ascending and descending chromatic progressions. Stimuli 5 and 6 present ascending and descending melodic sequences (in the musical sense of the term), which differ from stimuli 3 and 4 in their complexity, since their ascending or descending motions are not confined to the most immediate melodic level. The effect of interval size was examined in stimuli 7-10 such that stimuli 7 and 8 present increasing and decreasing melodic intervals in ascent, while stimuli 9 and 10 present increasing and decreasing melodic intervals in descent. Stimuli 11 and 12 typify an accelerando versus a ritardando, and stimuli 13 and 14 present an increase versus a decrease in motivic pace. Stimuli 15 and 16 present gradual articulation changes (from tenuto to staccatissimo and vice versa) of a steady pitch in unchanging IOI. Stimuli were created through Sibelius 1.2 music software, using the software’s Grand Piano sound, with the software’s “expression” and rubato features turned off. They were recorded onto an audio CD using two identical tracks (i.e., a monophonic recording). Experiment 2 PARTICIPANTS Ninety-five Tel Aviv University students (67 females, 28 males; mean age ⫽ 25.3, SD ⫽ 6.54) participated in the experiment. Thirty-five of these (21 females, 14 males) had at least 7 years of music training (“musicians”), while the remaining 60 had little or no formal music training (46 females, 14 males). TASK AND PROCEDURE These were identical to those of Experiment 1. However, one item was added to the forced-choice questionnaire. While in Experiment 1 we asked participants to evaluate the overall “energy” of the imagined motion (item 8), in Experiment 2 we asked them, in addition, to rate the change in the imagined motion’s energy level (comparing energy levels in the motive’s beginning and ending) on a scale of –3 to ⫹3. A condition in which participants imagined their character moving without any accompanying musical figure (and filled out their questionnaire accordingly) was added to this experiment. We included this condition to determine whether any a priori motion imagery tendencies (e.g., a tendency to imagine ascending rather than descending motion) exist, regardless of any musical stimulation. If such tendencies were found, they (rather than a random distribution of the responses to each question) would have served as a baseline, against which results in each musical condition would be evaluated. EXPERIMENTAL MATERIALS Musical stimuli were identical to those of Experiment 1 but were slowed down from 160 BPM to 90 BPM. Statistical Methods EXPERIMENTS 1 AND 2 Answers to the ordered multiple-choice items (questions 2–7; see method section) were coded as –1 or 1 (for opposing answers) and as 0 for a neutral answer (e.g., in the verticality item, “ascending” is ⫹1, “descending” is –1, and “level” is 0). Items 8 (level of energy) and 9 (change in the level of energy, Experiment 2) were analyzed as continuous variables. Most of the statistical analyses are based on differences between the coded answers to paired musical motives representing an intensifying figure and its corresponding abating figure (e.g., no. 1, a crescendo, as compared to no. 2, a diminuendo). Paired comparisons for each question were carried out using the Wilcoxon test on the differences in the coded answers to the two members of each pair of motives. The latter tests were conducted on the entire sample and separately for musicians and nonmusicians. A Wilcoxon test for independent samples indicated whether differences between musicians and nonmusicians were significant. For questions with opposing answers, chi-square tests were used for each motive to test the hypothesis that non-neutral responses favored one of the two directions (e.g., right vs. left motion). In addition, in each pair of motives the sign test was used to test whether such tendencies were stronger in the intensifying than in the abating motive. The question on type of motion (q. 1) was analyzed as a categorical response variable using the McNemar test for correlated proportions (see, e.g., Hays, 1973, pp. 741-742) to compare the proportion of subjects who chose specific answers in the intensifying motives as compared to the abating motives. 04.MUSIC.23_221-248.qxd 01/02/2006 12:26 Page 229 How Music Moves: Musical Parameters and Listeners’ Images of Motion 229 FIG. 1. The 16 melodic figures (eight pairs) used in the two experiments. One member of each pair presents an “increase” (intensification) in a specific musical parameter, while the other presents a “decrease” (abatement). The chi-square and sign tests described above were also applied to groups of motives that share a particular musical characteristic (e.g., all rising motives, 3, 5, 7, and 8). Wilcoxon tests compared results for groups of motives that differ with regard to a particular character- istic (e.g., all rising motives vs. all falling motives, 4, 6, 9, and 10). In these tests, coded answers for an entire group were derived, for each participant, by summing up the coded answers for all motives in the group. For instance, for question 2, the response of a participant who 04.MUSIC.23_221-248.qxd 230 01/02/2006 12:26 Page 230 Z. Eitan and R. Y. Granot FIG. 1. Continued answered “ascending” to three of four stimuli in a group was considered “ascending.” A neutral (0) code was assigned to participants whose sum of responses showed no preference for either opposites (i.e., two “ascending” responses and two “descending” responses). Note that the set of responses given by the same subject are dependent. We exploit this fact in the group analysis by computing summary measures across several responses (as explained above) and carrying out statistical analyses on these summary measures. As in each group of stimuli there is just one summary measure for each subject, these results are statistically independent. The results of Experiment 1 (fast tempo) and Experiment 2 (slow tempo) were compared by way of a chi-square test on the distribution of responses to each of the ordered multiple-choice items for each motive. 04.MUSIC.23_221-248.qxd 01/02/2006 12:26 Page 231 How Music Moves: Musical Parameters and Listeners’ Images of Motion The false discovery rate procedure (Benjamini & Hochberg, 1995) was used to account for multiple testing. Results The main statistical results are presented in five tables, summarizing significance values in the two experiments. In addition, a sample of the actual data is presented in bar figures (Figures 2–4). In each figure, we present responses associated with motion related to the vertical direction (A), distance change (B), lateral direction (C), and speed change (D) in the fast-tempo Experiment 1 (A1, B1, C1, D1) and in the slow-tempo Experiment 2 (A2, B2, C2, D2). Tables 1 and 2 exhibit the relationships between musical parameters and motion features in the two experiments (* indicating significance levels in Experiment 1, ⫹ indicating significance levels in Experiment 2). Table 1 is based on results of Wilcoxon Signed Ranks tests. As mentioned, these tests compared responses to each forced-choice question in contrasting pairs of motives (e.g., whether responses to the two motives contrasting in dynamics, a crescendo and a diminuendo, differ with regard to the lateral direction of the imagined character). Thus, the table indicates which musical and motional parameters are significantly associated. In particular, it shows that some musical parameters such as dynamics and pitch contour are associated with many aspects of imagined motion, and others, such as pitch intervals, are more limited in their impact. Similarly, it points to motional features 231 (e.g., speed change) associated with diverse musical parameters and to those (e.g., lateral direction) that are only affected by a single musical parameter. Table 2 presents significance values of chi-square tests, indicating differences between opposing answers (e.g., right vs. left) for each motive separately. The information in Tables 1 and 2 is complementary. For instance, while Table 1 generally indicates that dynamics affects the imagined vertical direction (top left cell), corresponding chi-square results (Table 2, top left cells) show that it is only diminuendo, rather than crescendo, that is significantly associated with verticality. In addition, chi-square tests for individual motives may provide interesting information even where results of Wilcoxon tests for the relevant motive pair are not statistically significant. For instance, a Wilcoxon test shows that changes in IOI are not significantly associated with imagined vertical direction (Table 1, row 4). Yet, as indicated by chi-square tests (Table 2), the similarity between the two figures does not stem from their lack of effect on verticality but from the fact that both are significantly associated with the same vertical direction (descent). In the following text, we survey the main motional features associated with each musical parameter in the two experiments, as specified in Tables 1 and 2. Dynamics As seen in Table 1 (top row), and in Figure 2 in both experiments, changes in dynamics (within the context of a repeating tone) significantly affect, as expected, the distance of the imagined figure (Figure 2: B1, B2). TABLE 1. Motion features associated with musical parameters (Wilcoxon signed ranks tests) Z-scoresa Vertical direction Dynamics Pitch contour Pitch intervals IOI Distance change ⫺3.124*** ⫺3.67151⫹⫹⫹ ⫺5.929**** ⫺5.16596⫹⫹⫹⫹ ⫺7.078**** ⫺7.47089⫹⫹⫹⫹ ⫺3.02** Motivic pace a ⫺4.382**** ⫺3.6043⫹⫹⫹ Speed change External force ⫺5.179**** ⫺3.72949⫹⫹⫹ ⫺2.3094⫹ ⫺3.204*** ⫺4.41289⫹⫹⫹⫹ ⫺2.123* Energy level Energy change (Exp. 2 only) ⫺2.817** ⫺3.0319⫹⫹⫹⫹ ⫺4.57539⫹⫹⫹⫹ ⫺2.456* ⫺4.66931⫹⫹⫹⫹ ⫺5.75778⫹⫹⫹⫹ ⫺2.05* ⫺2.98165⫹⫹ Articulation Lateral direction ⫺3.784**** ⫺3.38085⫹⫹⫹ ⫺6.36**** ⫺5.4164⫹⫹⫹⫹ ⫺5.25**** ⫺2.58567⫹⫹ ⫺4.47666⫹⫹⫹⫹ ⫺3.28077⫹⫹ ⫺3.04764⫹⫹ and their respective 2-tailed levels of asymptotic significance In Experiment 1 *p ⬍ .05; **p ⬍ .01; ***p ⬍ .001; ****p ⬍ .0001. In Experiment 2 ⫹p ⬍ .05; ⫹⫹p ⬍ .01; ⫹⫹⫹p ⬍ .001; ⫹⫹⫹⫹p ⬍ .0001. ⫺2.56817⫹⫹⫹⫹ 04.MUSIC.23_221-248.qxd 232 01/02/2006 12:26 Page 232 Z. Eitan and R. Y. Granot TABLE 2. Motion features associated with musical parameters (chi-square) Vertical direction Ascent Dynamics Crescendo Rise Fall Pitch intervals Increasing Decreasing IOI Accelerating Decelerating Motivic pace Approach Away Lateral direction Right Left 16.75**** 34.71⫹⫹⫹⫹ Diminuendo Pitch contour Descent Distance change 18.24**** 42.12⫹⫹⫹⫹ Slower 65.21**** 67.16⫹⫹⫹⫹ 11.79⫹⫹⫹b 13.09*** 6.7⫹⫹ 7.11** 9.52⫹⫹ 8.0** External force Support Negate 19.6⫹⫹⫹⫹ 12.75⫹⫹⫹ 20.54**** 8.56⫹⫹ 20.64**** 19.26⫹⫹⫹⫹ 9.96** 11.76*** 9.98⫹⫹ 9.68⫹⫹b 6.72** 8.34⫹⫹ 25.08**** 54.08⫹⫹⫹⫹ 5.48* 15.2⫹⫹⫹ 38.62**** 60.84⫹⫹⫹⫹ Accelerating Decelerating Faster 34.88**** 44.64⫹⫹⫹⫹ 33.37**** 55.84⫹⫹⫹⫹ 8.06** 5.72⫹a 11.2*** 13.12⫹⫹⫹ Speed change 5.76* 48.28**** 44.8⫹⫹⫹⫹ 16.02**** 6.72** 18.64⫹⫹⫹⫹ Articulation Toward tenuto Toward staccato 6.42* 6.12⫹ 10.93*** 12.16⫹⫹⫹ 6.76** 15.125⫹⫹⫹⫹ Note. df ⫽ 1. a Though in both experiments rising motives as a whole tend to be associated with motion away, stimulus 5 in Experiment 2 (an ascending melodic sequence) was associated with approaching motion (p ⬍ .01). bIn Experiment 2, stimuli involving progressive changes in pitch intervals tend to be associated with spatial fall and with motion away, regardless of the direction (increase or decrease) of interval change. In Experiment 1: *p ⬍ .05. **p ⬍ .01. ***p ⬍ .001. ****p ⬍ .0001. In Experiment 2: ⫹p ⬍ .05. ⫹⫹p ⬍ .01. ⫹⫹⫹p ⬍ .001. ⫹⫹⫹⫹p ⬍ .0001. However, dynamics also influences imagined changes in verticality and speed (Figure 2: A1, A2; D1, D2), as well as the level of energy associated with the motion of the imagined figure. In the slower stimuli of Experiment 2, dynamics also affects, in addition to the above, the tendency to ascribe motion to the impact of an outside force on the imagined character, and the direction (increase or decrease) of energy change. As indicated by chi-square tests for each stimulus (Table 2), in both experiments crescendi tend to evoke approaching and accelerating motion (but not an ascent), while diminuendi are associated with moving away and with descending motion. Diminuendi are also associated with slowing down, though only in Experiment 2, when overall tempi are slower. Crescendi and diminuendi also differ (as McNemar tests indicate) with regard to the types of movements they tend to be associated with, the former more frequently associated with running (though only in Experiment 1, p ⬍ .01), and the latter with falling or sliding (p ⬍ .01 in Experiment 1, p ⬍ .0001 in Experiment 2). Pitch Contour Results for pitch contour were computed with regard to all four rising stimuli combined (a ascending chromatic line, an ascending chromatic sequence, an ascent with increasing pitch intervals, and an ascent with decreasing intervals), as compared to all four descending stimuli (see also “Statistical Methods” above). In both experiments, motion images evoked by pitch “rises” and “falls” differ with regard to the dimensions of vertical direction, distance change (though only in Experiment 1), lateral direction, speed change, and energy level (see Table 1, second row from top, and Figure 3). As seen in Table 2, pitch “rises” suggest, as expected, spatial ascent. In addition they are significantly associated with moving away (contrary to the Doppler effect), with acceleration, and with higher energy. Pitch “falls,” on the other hand, are associated, as expected, with spatial descents, with slowing down (though as with the effects of the diminuendi, only in Experiment 2), and with lower energy, but they are not significantly associated with 04.MUSIC.23_221-248.qxd 01/02/2006 12:26 Page 233 How Music Moves: Musical Parameters and Listeners’ Images of Motion 233 FIG. 2. The influence of dynamics on features of imagined motion: vertical direction (rise, fall, no change), distance change (approach, away, no change), lateral direction (right, left, no change), and speed change (faster, slower, no change). Bars represent the distribution of responses to crescendo (bars on the left) as compared to diminuendo (bars on the right) among all participants in Experiment 1 (top row, N ⫽ 78) and the slowtempo Experiment 2 (bottom row, N ⫽ 95). FIG. 3. The influence of pitch contour on features of imagined motion: vertical direction (rise, fall, no change), distance (approach, away, no change), lateral direction (right, left, no change), and speed (faster, slower, no change). Bars represent the distribution of responses to ascending (bars on the left) as compared to descending contours (bars on the right) among all participants in Experiment 1 (top row, N ⫽ 78) and the slow-tempo Experiment 2 (bottom row, N ⫽ 95). approaching motion. Pitch falls are also strongly associated with motion to the left, while pitch rise is not significantly associated with rightward motion. In addition, pitch rises and falls tend (in both experiments) to evoke different types of motion (as indicated by McNemar tests): pitch rises suggest running or walking, pitch falls suggest falling motion (p ⬍ .001). Note also that in Experiment 1 both pitch rise and pitch fall are associated with motion supported by an external force. 04.MUSIC.23_221-248.qxd 234 01/02/2006 12:26 Page 234 Z. Eitan and R. Y. Granot Pitch Intervals To control for contour effects, results for the two stimuli presenting increasing intervals, rising and falling, were combined in a way similar to that described for pitch contour (see “Statistical Methods,” p. 228 above), as were results for rising and falling stimuli presenting decreasing intervals. Changes in pitch intervals were significantly associated only with distance change: decreasing intervals tended to strengthen a tendency to imagine motion away from the listener. This relationship, however, was significant only in Experiment 1 (Table 1). In Experiment 2, no significant relationship was found between the direction of interval change (increasing or decreasing) and any of the motion features investigated. However, stimuli presenting changes in pitch intervals—both increasing and decreasing—tend to be associated with spatial descent and with motion away in this experiment (Table 2). These results were carried mainly by the stronger association of descending stimuli with imagined spatial descent and with motion away from the listener, as compared to much weaker associations between rising stimuli and these two motional features. The change in interval size itself did not contribute significantly to these effects. Inter-Onset Intervals (IOIs) In both experiments, motion images evoked by changes in IOI differ, as expected, with regard to changes in imagined speed (Figure 4: D1, D2). In addition, both acceleration (decreasing IOI) and deceleration (increasing IOI) are associated, in Experiment 1, with images of descent (Table 2, Figure 4: A1, A2). This pattern is also evident in Experiment 2, although in this experiment accelerations only show a tendency to elicit images of descent. Chi-square analyses also suggest associations between decelerations and motion away from the listener (in Experiment 1). Although changes in motivic pace may be conceptualized as changes in IOI, the results presented in Tables 1 and 2 suggest that, at least with regard to the evoked motional images, changes in IOI within a repeating tone are not identical to changes in a more complex pattern. Nonetheless, some effects can be seen across the two musical contexts, especially so when the overall tempo is fast (Experiment 1). Articulation The last musical parameter we examined, articulation (transition from staccato to tenuto or vice versa) in a repeating tone context, is strongly associated in both experiments with change in distance (Table 1). This is primarily due to the association of a gradual increase in staccato with moving away (Table 2). Changes in perceived loudness concomitant with changes in articulation could underlie this effect. Chi-square analyses suggest some additional associations of articulation changes, though these were not replicated in both experiments (see Table 2). FIG. 4. The influence of tempo change on features of imagined motion: vertical direction (rise, fall, no change), distance (approach, away, no change), lateral direction (right, left, no change), and speed (faster, slower, no change). Bars represent the distribution of responses to acceleration (bars on the left) as compared to deceleration (bars on the right) among all participants in Experiment 1 (top row, N ⫽ 78) and the slow-tempo Experiment 2 (bottom row, N ⫽ 95). 04.MUSIC.23_221-248.qxd 01/02/2006 12:26 Page 235 How Music Moves: Musical Parameters and Listeners’ Images of Motion Asymmetries An especially interesting aspect of the data is that of the asymmetries in the spatio-kinetic associations related to opposing changes in a musical parameter. These asymmetries (see “directional symmetry hypothesis,” p. 226 above) are exhibited in Table 3, based on a sign test examining directional asymmetry. Dynamics, pitch, and tempo all demonstrate such asymmetries in several ways. Crescendi and diminuendi are directionally asymmetrical with regard to verticality: while diminuendi are strongly associated with a descending spatial motion, crescendi are not significantly associated with ascents. This asymmetry is highly significant in Experiment 2 and marginally significant in Experiment 1. In Experiment 1, crescendi and diminuendi are also asymmetrical with regard to speed: while crescendi evoke speeding up, diminuendi do not evoke slowing down. Directional asymmetries are particularly manifest with regard to pitch contour, whose spatiokinetic associations are asymmetrical with regard to several dimensions: speed change, distance change, horizontal direction, and perhaps most surprisingly, verticality. Thus, though pitch descents (as might have been expected) strongly evoke spatial descents, pitch rises are associated with spatial ascents to a considerably lesser degree, and for nonmusicians, not at all. This asymmetry is highly significant in both experiments. In addition, in both experiments pitch rise is associated with speeding up and approaching, while pitch fall is not associated with slowing down or moving away. Pitch 235 rise, however, is only weakly related to motion rightward, while falls strongly evoke turning to the left. Finally, in both experiments IOI changes are asymmetrical with regard to verticality, since both directions of IOI change (acceleration and deceleration) evoke descent. Music Training Table 4 summarizes the differences in the motion associations as a function of music training as obtained by Wilcoxon tests. In general, significant differences between the two groups are evident in only a few of the musical–motional relationships, all of which indicate a stronger music–motion association for musicians. These pertain to the association of pitch contour with verticality and laterality (in both experiments); of motivic pace with verticality (in Experiment 1) and with energy change (in Experiment 2); of inter-onset intervals with verticality and speed change (in Experiment 2); and of articulation with speed change (in Experiment 2). Note that no training-related differences were found with regard to the musical parameters of dynamics and melodic intervals. In contrast, training-related differences concerning the spatio-kinetic associations of pitch contour were the only ones significant in both experiments. Fast vs. Slow Tempo Table 5 displays the significant differences between participants’ kinetic associations for each of the 16 motives in their fast (Experiment 1) and slow (Experiment 2) TABLE 3. Asymmetries (sign test) Vertical direction Dynamics Distance change Lateral direction crescendo diminuendo Speed change **(1) (*)(2) ⫹⫹⫹ Pitch contour(3) rise fall IOI (**)(4) ** ⫹⫹ (⫹⫹)(5) **** **** ⫹⫹⫹ ⫹ accelerating ****(6) decelerating ****(6) ⫹ ⫹ In Experiment 1 *p ⬍ .05; **p ⬍ .01; ***p ⬍ .001; ****p ⬍ .0001. In Experiment 2 ⫹p ⬍ .05; ⫹⫹p ⬍ .01; ⫹⫹⫹p ⬍ .001; ⫹⫹⫹⫹p ⬍ .0001. (1) Meaning crescendo is more strongly related to speed change as compared to diminuendo. (2) p ⫽ .035, which is marginally significant, due to FDR correction. (3) Most results for pitch contour combine data from all relevant stimuli (four ascending and four descending). Results applying to a single pair of stimuli only are presented in brackets. (4) In Experiment 1, asymmetry in distance change is significant only with regard to stimuli 5 & 6 (chromatic sequences). (5) In Experiment 2, asymmetry in speed change is significant only with regard to stimuli 5 & 6 (chromatic sequences). (6) Both directions of IOI change suggest spatial descent. 04.MUSIC.23_221-248.qxd 236 01/02/2006 12:26 Page 236 Z. Eitan and R. Y. Granot TABLE 4. Musicians versus nonmusicians (Wilcoxon test for independent samples—Wilcoxon w) Vertical direction Dynamics Pitch contour Distance change 1107*** 2361⫹⫹⫹⫹ Pitch intervals IOI Motivic pace Lateral direction Speed change Energy Energy change (Exp. 2 only) 1302*** 2554⫹ 1233⫹⫹ 1217⫹⫹⫹ 1311.5** 1343.5⫹⫹ Articulation 2435⫹ 2258.5⫹⫹ In Experiment 1 *p ⬍ .05; **p ⬍ .01; ***p ⬍ .001; ****p ⬍ .0001. In Experiment 2 ⫹p ⬍ .05; ⫹⫹p ⬍ .01; ⫹⫹⫹p ⬍ .001; ⫹⫹⫹⫹p ⬍ .0001. TABLE 5. Fast (Experiment 1) versus slow tempo (Experiment 2) Vertical direction Dynamics Lateral direction Speed change Force 10.804**a 2265.5**** 3. Chromatic rise 4. Chromatic fall 2586.5** 5. Sequential rise 6.373* 6. Sequential fall Pitch intervals Energy 1. Crescendo 2. Diminuendo Pitch contour Distance change 1821.5**** 2233.5**** 7. Increasing (ascent) 8. Decreasing (ascent) 9. Increasing (descent) 10. Decreasing (descent) IOI 2448.0*** 11. Accelerating 12. Decelerating Motivic pace 14. Decelerating Articulation 2589.5** 13. Accelerating 2296.5**** 10.331** 15. Toward tenuto 16. Toward staccato 17.814**** 2673.5** 7.151* 2815.5* 11.445** 2694.5* Note. Mann-Whitney U is used for energy. Chi-square is used for all other motion features (df for vertical, distance, lateral, speed ⫽ 2; df for force ⫽ 1). a Two cells (33.3%) have expected count of less than 5. The minimum expected count is 3.33. *p ⬍ .05. **p ⬍ .01. ***p ⬍ .001. ****p ⬍ .0001. versions, as obtained by a direct comparison (using chisquare tests) between the two experiments. Predictably, the most conspicuous difference between the fast and slow versions is in the level of “energy” ascribed by participants to their imagined character. Overall faster tempo increases the energy level ascribed to imagined agents in 11 of 16 motives; the remaining five motives exhibited no significant differences in this regard. A few other significant differences were found. They include (a) a stronger inclination of diminuendi in an overall slower tempo to imply descent, (b) a stronger inclination in the slower tempo of sequential pitch rise to suggest interference by an external force, and (c) a stronger tendency in the faster tempo to associate pitch rises and acceleration. Decelerating motivic pace, in the slower tempo, has lost its association with spatial descent and depicts a speeding, rather than a slowing down. Finally, articulation change from tenuto to staccato did not evoke motion to the left in the slower tempo, as it did in the faster one, while a change from staccato to tenuto tends to be associated with slowing down in slower tempo. “No-Music” Condition When imagined motion was not related to any musical stimulus, responses yielded only one statistically 04.MUSIC.23_221-248.qxd 01/02/2006 12:26 Page 237 How Music Moves: Musical Parameters and Listeners’ Images of Motion significant result: participants tended to imagine motion as ascending rather than descending (chi-square ⫽ 6.53 [df ⫽ 1], p ⬍ .05). This result, however, was not replicated in two consecutive experiments we ran (Eitan & Granot, 2005c). Discussion Musical Parameters Affect Motion Imagery Strongly and Diversely All musical parameters investigated significantly affected parameters of motion imagery (some more than others), and all parameters of imagined motion were affected by music parameters (again, some more than others). Note that the auditory parameter least affecting motion imagery was melodic interval, an exclusively musical parameter, while parameters related to auditory experience in general, like dynamics (loudness) or pitch contour, were strongly associated with diverse aspects of motion. Relationships of Specific Musical and Motional Parameters The present experiments corroborate several of the hypotheses concerning particular musical–motional relationships. In both experiments, aspects of timing (IOI and motivic pace) were strongly related to speed (as suggested, for example, by Juslin, Friberg, & Bresin, 2002, or Repp, 1998); aspects of pitch contour were related to verticality, in line with the Western tradition of notation and musical discourse (Cox, 1999; Scruton, 1997); and aspects of loudness were related to distance and energy. In contrast, the expected Doppler-like relationship of pitch and distance (Neuhoff & McBeath, 1996) was reversed: pitch rise was associated with increasing distance from the observer, and pitch fall, with decreasing distance. These various associations notwithstanding, the overall picture emerging from this study is that imagined musical space is much more complex than that implied by entrenched analogies: deep-rooted associations, such as that of verticality and pitch contour, are intricately and asymmetrically structured (see below) and previously unacknowledged relationships (such as that of dynamics and verticality) prove to be as important as established ones. Intensity Direction Often Matches Musical and Motional Parameters Given the cross-modal nature of the percept of intensity (for a survey of relevant studies, see above, pp. 224–225, 237 and Eitan & Granot, 2005a), it is not surprising that, overall, intensifications in musical parameters are indeed associated with intensifying motion, while musical abatements are related to abating or passive motions. Thus, the “energy level” and the direction of “energy change” ascribed by listeners to imagined motion nearly always correlate with the hypothetical intensity direction of the associated musical motive: pitch ascents, as compared to descents, crescendi, as compared to diminuendi, and accelerandi, as compared to ritardandi (in Experiment 2), are associated with higher and increasing energy. Furthermore, intensity correlations, as we predicted, seem to generate one-to-many mappings, in which predictable musical–motional associations “spill over” to less obvious ones. For instance, pitch rises are related not only to spatial ascents but also to increase in velocity, though there is no IOI change in the musical stimuli. Crescendo also suggests, in addition to the expected approaching motion, increasing velocity, though again IOIs are isochronous. Indeed, in their free verbal descriptions, several participants described the motion accompanied by a crescendo as a walk turning into a run, or as a walk terminated by a leap. Diminuendo is associated not only with motion away from the listener but also with descent, though there is no pitch change in the musical stimulus. In participants’ free descriptions, the most frequent description accompanying diminuendi was of a fall, rather than moving away. Ritardando evokes a slowing motion but also suggests moving away from the listener, though dynamics (the musical parameter naturally associated with distance) remains constant. Multiple Mapping Strategies? Some significant musical–motional mappings, however, cannot be easily explained in terms of intensification relationships alone. For example, pitch tends to be associated with horizontal motion, that is, rises in pitch evoke an imagined motion to the right, while falls evoke an imagined motion to the left. Pitch rises, as mentioned, move away from rather than toward the listener. Decreasing pitch intervals are (in Experiment 1) also related to increasing distance, and musical acceleration (an increase, or intensification, in temporal density) is related to descent (associated with decreasing tension and effort). These mappings suggest that listeners do not necessarily use an all-encompassing mapping strategy (such as intensification analogy) for all stimuli. Rather, different parameters (or even specific stimuli) may suggest different mapping strategies. Some mappings may be based on intensity isomorphism or on other analogies 04.MUSIC.23_221-248.qxd 238 01/02/2006 12:26 Page 238 Z. Eitan and R. Y. Granot between visual and auditory “space.” For instance, mapping pitch intervals into perceived visual perspective (where distant objects seem “smaller”) may account for the association of decreasing intervals with moving away. The mapping of pitch into lateral direction may be another case in point, as this mapping may be an indirect outcome of the association found between right and up (both representing an increase as in a simple x, y graph) versus left and down (Weeks & Proctor, 1990; Cho & Proctor, 2005). If pitch is mapped into spatial verticality, and spatial verticality mapped into laterality, then higher pitch would be associated with motion to the right. Yet, this association may have been also affected by the structure of the piano keyboard, as Stewart, Walsh, and Frith (2004) suggest. This hypothesis is supported by the more decisive results of musicians in this respect: though only a few of the musicians were professional pianists, most played the instrument regularly. Such associations can be related to findings reported by Ashley and Joichi (2004), in which the concepts of pitch in various musical cultures seem to be related, among other things, to the structure of musical instruments central to the culture. Finally, the association of an accelerating repeated pitch with descent may also be iconic and particular, such as imagining a ball bouncing downhill, as suggested by some of our participants in their free descriptions. Indeed, the use of conflicting strategies may have contributed to the fact that both acceleration and deceleration were associated with descent, as participants who associated acceleration with descent responded to a concrete image (notably, the verb most frequently used in the free descriptions associated with the acceleration motive was “Hitdarderut,” meaning uncontrolled rolling down a slope), while those who related deceleration with descent might have reacted to the decreasing intensity connotations shared by these two gestures (with free descriptions indeed stressing declining energy or passivity).6 Though the above suggestions are speculative and need to be corroborated by further research, the present study does suggest two related observations: first, that images of space and motion affected by music may be constructed through diverse, sometimes conflicting mapping strategies both within and across listeners; and second, that this 6 A fact that may further complicate the association of rise and fall with increase and decrease under intensity is that bodily fall, associated with (and often stemming from) abatement, produces intensifying consequences (acceleration, increasing kinetic energy), while bodily rise, associated with intensification, often produces abating consequences (deceleration, decreasing kinetic energy, fatigue). diversity notwithstanding, with regard to most stimuli, listeners’ choices present clear, statistically significant tendencies both within and between experiments. Musical-Kinetic Analogies Are Often Directionally Asymmetrical Perhaps the most surprising finding of these experiments is the refutation of the directional symmetry hypothesis (see p. 226). Imagined musical space proves to be asymmetrical in diverse domains, as listeners who associate a musical stimulus with a particular kinetic quality often do not associate the inverse stimulus with the opposite kinetic quality. Thus, as mentioned, diminuendi descend, but crescendi do not ascend. Crescendi, however, speed up, while diminuendi (in faster tempi) do not slow down. Correspondingly, when pitch rises it moves faster (as well as further), but as it falls, it does not slow down or draw nearer. Pitch fall, however, moves strongly to the left, while pitch rise is only weakly related to motion rightward. Even the ingrained association of pitch and verticality is strongly asymmetrical: though for nearly everybody pitch indeed falls, for many (particularly for nonmusicians) it does not really rise. Indeed, a surprising number of participants chose to stress in their free descriptions that the motion accompanying the “rising” pitch motive was a spatial fall (e.g., “falling off a stairway,” “rolling down a slope”). In general, while musical abatements (pitch descents, diminuendo, ritardando) are strongly associated with spatial descents, musical intensifications (crescendo, accelerando, even pitch rise itself) are not as strongly associated with spatial ascent. Similarly, while musical intensifications (rise in pitch, crescendo, accelerando, increasing motivic pace, sequential melodic progressions) are generally associated with increasing velocity, musical abatements (pitch descents, diminuendo, decreasing motivic pace) are not generally associated with decreasing velocity. These asymmetries present an image of music-related space and motion that varies considerably from neatly symmetrical, rationalistic models of musical space, including the global intensity model proposed earlier in this article. Musical space, as suggested by these results, is not composed of simple binary oppositions but of differences.7 Opposite “directions” in this space do not present symmetrical contrasts (pitch rise is not the 7 The logic underlying notions of polarity and negation is in fact highly intricate and has been in debate from Aristotle to recent theories of pragmatics (see, e.g., Horn, 1989). 04.MUSIC.23_221-248.qxd 01/02/2006 12:26 Page 239 How Music Moves: Musical Parameters and Listeners’ Images of Motion opposite of pitch fall, and crescendo not the opposite of diminuendo), but rather, each stresses different spatial and kinetic associations. At the present stage, we can only speculate as to the source of these asymmetries. Two complementary approaches, however, seem to be promising. First, the notion of semantic markedness (Shapiro, 1976; see Hatten, 1994, for an application to music analysis) seems to be relevant to these results. Thus, for instance, in the present context spatial descent seems to be the unmarked term in the ascent-descent pair, as it is more frequently associated with most musical parameters (see Table 2, “vertical direction” column). Indeed, music theorists like Schenker (1935/1979) and recently Larson (1997) have related the descending progression to a natural tendency. Supporting force may also be unmarked, in relation to negating force (see “force” column in Table 2); moving away from the listener (perhaps perceived as “forward” from the imaginary character’s perspective) may be unmarked relative to moving toward the listener, and increasing speed may be unmarked relative to decreasing speed (“speed change” column, Table 2). In other words, it may be easier, more “natural” for listeners to associate musical parameters with one spatial polarity than with the other. Second, with regard to the effect of intensity change on bodily motion, “lower” may primarily contrast with “faster,” not with “higher.” While dwindling bodily energy often results in conspicuous lowering (falling, bending, sitting, or lying down), energy increase rarely makes us fly: its most conspicuous effect is on the speed and force of motion, rather than on vertical orientation. Consequentially, while abating musical stimuli were mostly interpreted by listeners as descents, intensifying stimuli were often not interpreted as ascents but as accelerations. Finally, the asymmetries reported here may be related to asymmetries in the distribution of musical parameters in actual musical pieces, specifically those in the Western repertory known to participants (see also below, pp. 241–242). For instance, some musical dimensions may tend to begin at a specific value (e.g., a relatively high pitch level) and hence deviate from that value more in one direction than in another (e.g., descend more than ascend). This preference for a given direction of change may reinforce the association of this direction with motion and weaken that of the opposite direction. Fast and Slow Tempi The considerable tempo difference between experiments 1 and 2 resulted in relatively few significant 239 differences in imagery. In general, then, dynamic changes in musical parameters seem to retain their effect on motion imagery within a wide range of tempi. This conclusion is noteworthy, since (as mentioned) tempo is the musical parameter most immediately associated with physical motion, and rhythmic-melodic figures are performed and perceived differently in different tempi (Repp, Windsor, & Desain, 2002; Windsor, Aarts, Desain, Heijink, & Timmers, 2001). This notwithstanding, some significant differences between the two experiments were found. The main difference, as reported above, concerns the lower “energy” ratings in the slower Experiment 2—obviously, not a great surprise. Other differences suggest tendencies that, though not central to the topic of this article, are interesting and worth noting. First, when overall tempo is slower (as is the case in Experiment 2, as compared to Experiment 1), listeners’ tendency to perceive motives as implying speeding up weakens, while their tendency to perceive implied slowing strengthens. This inclination is indicated by the significant difference between rising motives in the two experiments with regard to speed (rising motives in Experiment 1 are more strongly associated with speeding up). It is also suggested by the fact that decreases in loudness and pitch (as well as changes in articulation) are significantly related to slowing down only in the slower Experiment 2 (see Table 2). Thus, in the case of musical tempo, the perceived direction of parametric change seems to correlate with overall parametric values: when overall values are low (here, a slow tempo), the perception of decreasing changes (slowing down) is enhanced, while when overall values are high (here, fast tempo), perception of increasing changes is enhanced. It would be interesting to examine whether such correlations also apply to other parameters, within or outside the auditory domain. Second, differences between the experiments regarding motivic pace (see Table 5, stimulus 14) suggest that higher-level melodic and rhythmic patterns are difficult to perceive in slower tempi. While, on the surface, stimulus 14 exhibits an isochronous succession of 16th notes, on a slightly higher level it presents deceleration, as the motives composing it become progressively longer. This pattern also suggests descent in pitch, since the trough of each successive motive is a semitone lower than that of the preceding one (see Figure 1, stimulus 14). In Experiment 1, listeners responded to both these structural features by associating the stimulus with decelerating and descending motion. However, in the slower Experiment 2 neither of these associations is significant. Rather, there is a significant tendency to 04.MUSIC.23_221-248.qxd 240 01/02/2006 12:26 Page 240 Z. Eitan and R. Y. Granot perceive this stimulus as speeding up. These results are consistent with Bregman’s observation (1990; see also Van Noorden, 1975) that in primitive (bottom-up) streaming, within-stream integration and betweenstream segregation are enhanced as tempo increases. Thus, in the faster Experiment 1, listeners’ responses imply perceiving two different melodic streams within the stimulus, while in the slower Experiment 2 such stream segregation is weakened. pitch intervals—the former possibly because the spatiokinetic associations of loudness are strongly determined by its everyday, nonmusical connotations; the latter since melodic intervals, the centuries-old tradition relating them to musical gesture and affect notwithstanding, have had (within the limited scope of the present experiment) little gestural import, even for trained musicians. Some Caveats and Suggestions for Further Research The Effect of Music Training The most noteworthy finding of the comparisons of musically trained and untrained participants (in both experiments) was the scarcity of significant differences between the two groups. As noted above, in most respects musicians and nonmusicians relate musical and motional features in similar ways. Even where significant training-related differences exist, they do not indicate opposite tendencies for the two groups, but rather stronger tendencies by musicians to associate particular musical and motional parameters. It seems, then, that most aspects of musical–motional mappings do not stem from professional musical experience but from more general sources. Though musicians and nonmusicians do not use different musical–spatial mappings, the former seem to use such mapping more consistently and securely. While different factors can contribute to this finding (for instance, the wider and more extensive listening experience of musicians), it seems that musical practice, in particular the physical and motional experience of playing an instrument, sharpens the metaphorical faculties of practitioners, or at least crystallizes the reservoir of spatial and kinetic associations concerning musical parameters. It would thus be interesting to investigate whether musicians’ spatio-kinetic imagery is affected by the instrument in which they specialize. Musicians are particularly more consistent in their choice of spatial metaphors for pitch contour and temporal density (IOI, motivic pace). Thus, differences between the two groups apply, in both experiments, to the vertical and lateral associations of pitch height (possibly due to musicians involvement with musical notation and keyboards) and to the relationship of IOI and speed. Several significant differences (regarding speed and verticality in Experiment 1, and energy in Experiment 2) apply to motivic pace, probably since nonmusicians failed to perceive the “deeper-level” patterns of these stimuli (see “Fast and Slow Tempi” above). Note that musicians and nonmusicians do not significantly differ with regard to both dynamics and Both the task and the music-like materials used in the present study are highly constrained, and further studies are needed to assess whether their results apply to wider contexts. First, this experiment elicited rehearsed, created responses, expressed verbally. Its results thus primarily apply to a cognitive, decisional level, far removed from any basic perceptual relationships auditory and motional features may carry (though one should note that such relationships might have affected the responses indirectly). Though the level of processing on which the reported effects were generated does not directly influence our conclusions, it is clear that further work should be done in order to determine the level from which the relationships discovered here stem. Second, a task concerning imagining an external agent in motion, such as that used in the present experiment, differs in important respects from one involving self-motion imagery (which is intimately associated with musical gesture; see Todd, 1992). For instance, imagining self-motion would involve forward and backward movement, rather than change of distance.8 A third reservation may stem from the specific task used here, which concerns imagining an animated movie shot. Since music for animated film has traditionally included “Mickey-Mousing” music—iconic clichés for motion, synchronized with almost every movement (see, e.g., Prendergast, 1992)—some 8 Note that when the former task is substituted by the latter, motion away—an “abating” motion, associated with diminished visual image and sound—is replaced by forward motion, a common metaphor for intensification. In the present experiment, a subject’s identification with the imaginary screen character might have resulted in interpreting its movements as self-motion and thus interpreting moving “away” as moving “forward.” The surprising association of moving away with ascent may be explained by such interpretation of “away” as “forward,” since forward motion and ascent are both intensifying motions. Note also that to view a descending external object moving away, one must be situated above it. This constraint (which would not apply to self-motion) might have weakened the relationship between descent and moving away in the present experiment even further. Hence together these two factors may account for the association of ascent and moving away in this experiment. 04.MUSIC.23_221-248.qxd 01/02/2006 12:26 Page 241 How Music Moves: Musical Parameters and Listeners’ Images of Motion responses might have related the musical materials to those familiar musical icons, rather than applying a more general associative framework. To address this caveat, we have designed an additional experiment in which participants, who listened to stimuli identical to those presented in Experiment 1, were asked to imagine their own motion, rather than that of a cartoon character (Eitan & Granot, 2005c). A comparison of results of this “self-motion” experiment with those of Experiment 1 exhibits few significant differences. This suggests that task-driven associations from animated films did not play a major role in determining participants’ responses. Evident limitations of the musical materials used here should also be mentioned. First, like many cognitive studies, this study trades off ecological validity for a controlled environment, that is, it uses neither “real” musical motives nor a larger musical context, a context that might have affected the spatio-kinetic associations of specific motives. Furthermore, all stimuli constructed for the experiments present unidirectional changes in single parameters rather than the often noncongruent and complex parametric contours typifying gestures in actual music. Lastly, timbre, harmonic and melodic tonal structure, and most importantly metrical structure (a central musical factor in motion induction) were not investigated in this experiment. A number of follow-up experiments, altering task and materials, are thus needed in order to assess whether our findings apply to different contexts. We are currently in the process of performing or designing several such experiments. As mentioned, an experiment altering the imagery task involving imagined self-motion, rather than picturing an external movie shot, has already been performed. Another experiment completed recently examines the motional associations of congruent or noncongruent changes in the parameters studied here (e.g., a crescendo and an accelerando vs. a crescendo and a ritardando). Nonverbal tasks, in which participants may depict a moving object by drawing, by self-motion, or through a computer interface, are also planned. To test whether listeners’ motional associations, as obtained in this study, are evoked spontaneously or are only a by-product of the task, we also plan a comparison of memory performance in stimuli in which the musical and spatio-kinetic associations concur or diverge. Other experiments planned involve different stimuli, including tonal-functional motives (melodic and harmonic), stimuli set in a metrical framework, and actual musical materials set within larger musical contexts. Lastly, it would be worthwhile to examine the effects of culture and native language on listeners’ sound- 241 induced motion imagery. As noted in the introduction, different cultures relate spatial concepts and musical dimensions such as pitch in different ways (e.g., Ashley & Joichi, 2004; Zbikowski, 1998). Thus, broadening the scope of this study into non-Western musical cultures could shed light on the interplay between cultural phenomena and possible inherent or acquired perceptual biases. Specific effects of native language may also affect the results and should be examined. For instance, the reported responses concerning lateral direction may have been influenced by the participants’ native language, Hebrew, which is written from right to left (however, recent unpublished experiments by the first author and Renee Timmers suggest that the lateral associations of pitch reported here are shared by native English speakers). Implications for Music Theory and Music Technology Some possible implications of findings reported here (in particular the wide-ranging directional asymmetries) to music theory and music technology are noteworthy. 1. Statistical studies of diverse musical repertories indicate that several musical parameters are directionally asymmetrical in their distribution. Large ascending melodic intervals are more frequent than descending ones, while stepwise descents are more frequent than ascents (Vos & Troost, 1989). Dynamics, texture, and pitch contour tend to build up gradually but subside quickly (Huron, 1990a, 1990b, 1992). Relating such distributional asymmetries and the asymmetrical kinetic associations observed here may provide for some intriguing hypotheses. (For instance, are ascending leaps more frequent than descending ones since it is harder to convey vertical motion through pitch ascent than through descent? Or conversely, does the asymmetry in distribution affect, as suggested on p. 239 above, the asymmetry of motional associations?) 2. The metaphor of musical space is applied to diverse models in recent music theory, either directly, as in neo-Riemannian spaces (Lewin, 1987), Lerdahl’s pitch space (2001), or Morris’s contour space (1987); or indirectly, as in Narmour’s “parametric scales” (1990, 1992). Applying directional asymmetry to musical spaces or parametric scales, thus reflecting the findings reported here (e.g., weighting pitch rise and fall differently, in order to reflect the different strengths of their vertical and lateral implications), may provide them with higher perceptual feasibility. The findings reported here may also corroborate 04.MUSIC.23_221-248.qxd 242 01/02/2006 12:26 Page 242 Z. Eitan and R. Y. Granot models of pitch space that do apply directional asymmetry, such as that of Larson (1997), and implicitly also Schenker’s (1935/1979). 3. Music semioticians (Lidov, 1987; Cumming, 1997; Hatten, 1997-2002, 2005) suggest that musical gestures map expressive human motion onto musical features. The analysis of such mapping would need to take into account directional asymmetries. For instance, an interpretation in terms of gesture of symmetrical ascending and descending pitch structures (such as our chromatic rise and fall in Figure 1, stimuli 3 and 4) would map them into different, not contrasting gestures, with the descending pitch stressing vertical motion and the ascending stressing pitch energy and pace. Similarly, acknowledging the directional asymmetry of gestures may provide new insights into embodied melodic figures, such as the cross-cultural lament gestures (e.g., Mazo, 1994; Tolbert, 1990). 4. In recent years, increasingly sophisticated digital devices mapping human motion into music or vice versa have been developed (Camurri & Volpe, 2003; Wanderley & Battier, 2000). Surprisingly, such mapping is rarely based on perceptual or cognitive data, but rather on personal intuitions or unexamined shared suppositions. The present findings, particularly one-to-many correspondences and directional asymmetries reported, may help to establish musical–motional mapping algorithms on a more solid empirical base. Applying such findings to computational mapping algorithms may in turn provide an important testing forum for their validity. much more complex than hitherto assumed. Though some of the widely shared assumptions concerning music and motion have been corroborated here—most important, the hypothesis that listeners indeed map musical features into kinetic ones consistently—the cognitive mapping of music into motion is surprisingly multifaceted. Musical parameters associate simultaneously with several aspects of motion, not only those traditionally associated with them: a crescendo both approaches and accelerates motion; a pitch fall moves downward, leftward, and closer; and musical space seems to be skewed in many different ways, rather than composed of neatly arranged, symmetrical parametric scales and intervals. Discovering such complexity while using extremely simple stimuli suggests that an even more entangled, more challenging musico-kinetic web shapes actual musical materials and listening contexts. Author Note Conclusions We thank Richard Ashley, Bruno Repp, Henkjan Honing, and two anonymous referees for Music Perception for their thoughtful comments; David Steinberg and Yulia Gavrilov for their assistance in statistical analysis; and Noa Ravid-Arazi for her help in many practical facets of the experiments. Research for this article was supported by an Israel Science Foundation Grant no. 800/02-27.0. Findings reported in this article were presented at the conference “Music and Gesture,” University of East Anglia, August 2003; at the ESCOM Conference on Interdisciplinary Musicology, Graz, Austria, April 2004; and at the 8th International Conference for Music Perception and Cognition, Evanston, IL, August 2004. While asking simple questions and using simple musical stimuli to test them, this study has started to reveal that the cognitive mapping of music into space and motion is Address correspondence to: Zohar Eitan, Department of Musicology, Tel Aviv University, Tel Aviv, Israel, 69978. E-MAIL zeitan@post.tau.ac.il References ABRIL, C. (2001). The use of labels to describe pitch changes by bilingual children. Bulletin of the Council for Research in Music Education, 151, 31-40. AGAWU, V. K. (1982). The structural highpoint as determinant of form in nineteenth century music. Doctoral dissertation, Stanford University. ANGELL, J. R. (1906). Psychology: An introductory study of the structure and function of human consciousness. New York: Henry Holt. ASHLEY, R. (2003). Spatial elements of meaning in musicians’ body movements: Pragmatic and semantic dimensions. Paper presented at the International Conference “Music and Gesture,” University of East Anglia, Norwich, UK, August 28-30, 2003. ASHLEY, R. (2005). Cross-modal processing of melodies: Musical, visual, and spatial aspects. Manuscript submitted for publication. 04.MUSIC.23_221-248.qxd 01/02/2006 12:26 Page 243 How Music Moves: Musical Parameters and Listeners’ Images of Motion ASHLEY, R., & JOICHI, J. (2004). Musical pitch, spatial relationships, and language family: A cross-cultural investigation. In S. Lipscomb, R. Ashley, R. Gjerdingen, and P. Webster (Eds.), Proceedings of the 8th International Conference on Music Perception & Cognition (ICMPC8). Adelaide: Causal Productions. BAILY, J. (1985). Music structure and human movement. In P. Howell, I. Cross, & R. West (Eds.), Musical structure and cognition (pp. 237-258). London: Academic Press. BARKER, A. (1989). Greek musical writings: Vol. 2. Harmonic and acoustic theory. Cambridge: Cambridge University Press. BENJAMINI, Y., & HOCHBERG, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society, Series B, 57, 289-300. BERNSTEIN, I. H., & EDELSTEIN, B. A. (1971). Effects of some variations in auditory input upon visual choice reaction time. Journal of Experimental Psychology, 87, 241-247. BERRY, W. (1976). Structural functions in music. Englewood Cliffs, NJ: Prentice-Hall. BOLIVAR, V. J., COHEN, A. J., & FENTRESS, J. C. (1994). Semantic and formal congruency in music and motion pictures: Effects on the interpretation of visual action. Psychomusicology, 13, 122-154. BREGMAN, A. S. (1990). Auditory scene analysis: The perceptual organization of sound. Cambridge, MA: MIT Press. BROWER, C. (2001). Pathway, blockage, and containment in Density 21.5. Theory and Practice, 22/23, 35-54. CAMURRI, A., & VOLPE, G. (Eds.). (2003). Gesture-based communication in human-computer interaction. 5th International Gesture Workshop, GW 2003, Genova, Italy, April 15-17, 2003 Selected Revised Papers. Lecture Notes in Artificial Intelligence, Vol. 2915. Berlin: Springer Verlag. CHO, Y., & PROCTOR, R. (2005). Representing response position relative to display location: Influence on orthogonal stimulusresponse compatibility. The Quarterly Journal of Experimental Psychology, A, 58, 839-864. CLARKE, E. F. (2001). Meaning and the specification of motion in music. Musicae Scientiae, 5, 213-234. CLARKE, E. F., & DAVIDSON, J. (1998). The body in performance. In W. Thomas (Ed.), Composition–performance–reception. Studies in the creative process in music (pp. 74-92). Aldershot: Ashgate Press. CLYNES, M., & NETTHEIM, E. (1982). The living quality of music. In M. Clynes (Ed.), Music, mind and brain: The neurobiology of music. New York: Plenum Press. COHEN, D. (2001). The imperfect seeks its perfection: Harmonic progression, directed motion, and Aristotelian physics. Music Theory Spectrum, 23, 139-169. COLLIER, W. G., & HUBBARD, T. L. (2001). Judgments of happiness, brightness, speed and tempo change in auditory stimuli varying in pitch and tempo. Psychomusicology, 17, 36-55. 243 COOK, N. (1998). Analysing musical multimedia. Oxford: Clarendon Press. COSTA-GIOMI, E., & DESCOMBES, V. (1996). Pitch labels with single and multiple meanings: A study of French-speaking children. Journal of Research in Music Education, 44, 204-214. COX, A. (1999). The metaphoric logic of musical motion and space. Doctoral dissertation, University of Oregon. CUMMING, N. (1997). The subjectivities of Erbarme Dich. Music Analysis, 16, 5-44. DAVIDSON, J. W. (2002). Understanding the expressive movements of a solo pianist. Musikpsychologie, 16, 9-31. DAVIES, S. (1994). Musical meaning and expression. Ithaca: Cornell University Press. DESAIN, P., & HONING, H. (1993). Tempo curves considered harmful. Contemporary Music Review, 7, 123-138. EITAN, Z. (1997). Highpoints: A study of melodic peaks. Philadelphia: University of Pennsylvania Press. EITAN, Z., & GRANOT, R. (2003). Inter-parametric analogy and the perception of similarity in music. Proceedings of the 5th ESCOM Conference, Hanover. EITAN, Z., & GRANOT, R. (2005a). Intensity changes and perceived similarity: Interparametric analogies. Musicae Scientiae. EITAN, Z., & GRANOT, R. (2005b). [The effect of dynamic changes in musical parameters on listeners’ assessment of tempo change]. Unpublished raw data. EITAN, Z., & GRANOT, R. (2005c). [Musical parameters and motion imagery: Three follow-up experiments]. Unpublished raw data. FELDMAN, J., EPSTEIN, D., & RICHARDS, W. (1992). Force dynamics of tempo change in music. Music Perception, 10, 185-204. FLOWERS, P. J., & COSTA-GIOMI, E. (1991). Verbal and nonverbal identification of pitch changes in a familiar song by English and Spanish speaking preschool children. Bulletin of the Council for Research in Music Education, 101, 1-12. FREDRICKSON, W. E. (1997). Elementary, middle and high school student perceptions of tension in music. Journal of Research in Music Education, 45, 626-638. FREGO, R. J. D. (1996). The effect of aural, visual, and aural/visual conditions on students’ response to perceived artistic tension in music and dance. Doctoral dissertation, Florida State University. FRIBERG, A., & SUNDBERG, J. (1999). Does music performance allude to locomotion? A model of final ritardandi derived from measurements of stopping runners. Journal of the Acoustical Society of America, 105, 1469-1484. GABRIELSSON, A. (1973a). Adjective ratings and dimension analyses of auditory rhythm patterns. Scandinavian Journal of Psychology, 14, 244-260. GABRIELSSON, A. (1973b). Similarity ratings and dimension analysis of auditory rhythm patterns. I and II. Scandinavian Journal of Psychology, 14, 138-160, 161-176. 04.MUSIC.23_221-248.qxd 244 01/02/2006 12:26 Page 244 Z. Eitan and R. Y. Granot GABRIELSSON, A. (1999). Music performance. In D. Deutsch (Ed.), Psychology of music (2nd ed., pp. 501-602). New York: Academic Press. GARNER, W. R. (1974). The processing of information and structure. Potomac, MD: Erlbaum. GJERDINGEN, R. O. (1994). Apparent motion in music? Music Perception, 11, 335-370. HAIR, H. I. (1981). Verbal identification of music concepts. Journal of Research in Music Education, 29, 11-21. HANSLICK, E. (1986). On the musically beautiful (G. Payzant, Trans. and Ed.). Indianapolis: Hackett. (Original work published 1891) HATTEN, R. S. (1994). Musical meaning in Beethoven: Markedness correlation, and interpretation. Bloomington: Indiana University Press. HATTEN, R. S. (1997-2002). Musical gesture: On-line lectures. Cyber Semiotic Institute, University of Toronto. Retrieved from http://www.chase.utoronto.ca/epc/srb/cyber/ hatout.html HATTEN, R. S. (2005). Interpreting musical gestures, topics, and tropes: Mozart, Beethoven, Schubert. Bloomington: Indiana University Press. HAYS, W. L. (1973). Statistics for the social sciences (2nd ed.). New York: Holt, Rinehart, & Winston. HONING, H. (2003). The final ritard: On music, motion, and kinematic models. Computer Music Journal, 27, 66-72. HOPKINS, R. G. (1990). Closure in Mahler’s music: The role of secondary parameters. Philadelphia: University of Pennsylvania Press. HORN, L. (1989). A natural history of negation. Chicago: University of Chicago Press. HURON, D. (1990a). Crescendo/diminuendo asymmetries in Beethoven’s piano sonatas. Music Perception, 7, 395-402. HURON, D. (1990b). Increment/decrement asymmetries in polyphonic sonorities. Music Perception, 7, 385-393. HURON, D. (1992). The ramp archetype and the maintenance of passive auditory attention. Music Perception, 10, 83-92. IYER, V. (2002). Embodied mind, situated cognition and expressive microtiming in African-American music. Music Perception, 19, 387-414. JAQUES-DALCROZE, E. (1967). Rhythm, music, and education (H. F. Rubenstein, Trans.). London: The Dalcroze Society. (Original work published 1921). JOHNSON, M. L. (1987). The body in the mind. Chicago: University of Chicago Press. JOHNSON, M. L., & LARSON, S. (2003). “Something in the way she moves”: Metaphors of musical motion. Metaphor and Symbol, 18, 63-84. JUSLIN, P. N., FRIBERG, A., & BRESIN, R. (2002). Toward a computational model of expression in performance: The GERM model. Musicae Scientiae, Special Issue, 63-122. KIVY, P. (1980). The corded shell: Reflections on musical expression. Princeton: Princeton University Press. KIVY, P. (1990). Music alone: Philosophical reflections on the purely musical experience. Ithaca: Cornell University Press. KRONMAN, U., & SUNDBERG, J. (1987). Is the musical retard an allusion to physical motion? In A. Gabrielsson (Ed.), Action and perception in rhythm and music (pp. 69-80). Stockholm: Publications issued by the Royal Academy of Music, No. 55. KRUMHANSL, C. L. (1996). A perceptual analysis of Mozart’s piano sonata K. 282: Segmentation, tension, and musical ideas. Music Perception, 13, 401-432. KRUMHANSL, C. L., & SCHENCK, D. L. (1997). Can dance reflect the structural and expressive qualities of music? A perceptual experiment on Balanchine’s choreography of Mozart’s Divertimento No. 15. Musicae Scientiae, 1, 63-84. KUBOVY, M., & VAN VALKENBURG, D. (2001). Auditory and visual objects. Cognition, 80, 97-126. KURTH, E. (1971). Grundlagen des linearen Kontrapunkts [Foundations of Linear Counterpoint]. Hildesheim: G. Olms. (Original work published 1917) KURTH, E. (1969). Musikpsychologie. Hildesheim: G. Olms. (Original work published 1931) KURTH, E. (1991). Selected writings (L. A. Rothfarb, Trans. & Ed.). Cambridge: Cambridge University Press. LAKOFF, G., & JOHNSON, M. (1980). Metaphors we live by. Chicago: University of Chicago Press. LANGER, S. (1953). Feeling and form. London: Routledge. LARSON, S. (1997). Musical forces and melodic patterns. Theory and Practice, 22/23, 55-71. LERDAHL, F. (2001). Tonal pitch space. New York: Oxford University Press. LEWIN, D. (1987). Generalized musical intervals and transformations. New Haven: Yale University Press. LIDOV, D. (1987). Mind and body in music. Semiotica, 66, 69-97. LIDOV, D. (1999). Elements of semiotics. New York: St. Martin’s Press. LIPSCOMB, S. D. (in press). The perception of audio-visual composites: Accent structure alignment of simple stimuli. Selected Reports in Ethnomusicology, 12. LIPSCOMB, S. D., & KENDALL, R. A. (1994). Perceptual judgment of the relationship between musical and visual components in film. Psychomusicology, 13, 60-98. LIPSCOMB, S. D., & KIM, E. M. (2004). Perceived match between visual parameters and auditory correlates: An experimental multimedia investigation In S. Lipscomb, R. Ashley, R. Gjerdingen, and P. Webster (Eds.), Proceedings of the 8th International Conference on Music Perception & Cognition (ICMPC8), Evanston, IL, 3-8 August, 2004. Adelaide: Causal Productions. MADSEN, C. K., & FREDRICKSON, W. E. (1993). The experience of musical tension: A replication of Nielsen’s research using the 04.MUSIC.23_221-248.qxd 01/02/2006 12:26 Page 245 How Music Moves: Musical Parameters and Listeners’ Images of Motion continuous response digital interface. Journal of Music Therapy, 30, 46-63. MALLOCH, S. N. (2000). Communicative musicality and human companionship. Paper presented to the 6th International Conference on Music Cognition and Perception, Keele, UK, August 2000. MARKS, L. E. (1978). The unity of the senses: Interrelations among the modalities. New York: Academic Press. MARKS, L. E. (2000). Synesthesia. In E. A. Cardeña, S. J. Lynn, & S. C. Krippner (Eds.), Varieties of anomalous experience: Phenomenological and scientific foundations. Washington, DC: American Psychological Association. MAURER, D. (1993). Neonatal synesthesia: Implications for the processing of speech and faces. In B. de Boysson-Bardies, S. de Schoenen, P. Jusczyk, P. McNeilage, & J. Morton (Eds.), Developmental neurocognition: Speech and face processing in the first year of life (pp. 109-124). Dordrecht, Holland: Kluwer. MAZO, M. (1994). Lament made visible: A study of paramusical elements in Russian lament. In B. Yung & J. Lam (Eds.), Themes and variations: Writings on music in honor of Rulan Chao Pian (pp. 164-211). Harvard University, Dept. of Music, and the Institute of Chinese Studies, Chinese University of Hong Kong. MELARA, R. D., & MARKS, L. E. (1990a). Perceptual primacy of dimensions: Support for a model of dimensional interaction. Journal of Experimental Psychology: Human Perception and Performance, 16, 398-414. MELARA, R. D., & MARKS, L. E. (1990b). Interaction among auditory dimensions: Timbre, pitch, and loudness. Perception and Psychophysics, 48, 169-178. MELARA, R. D., & MARKS, L. E. (1990c). Hard and soft interacting dimensions: Differential effects of dual context on classification. Perception and Psychophysics, 47, 307-325. MELARA, R. D., & O’BRIEN, T. P. (1987). Interaction between synesthetically corresponding dimensions. Journal of Experimental Psychology: General, 116, 323-336. MEYER, L. B. (1989). Style and music: Theory, history, and ideology. Philadelphia: University of Pennsylvania Press. MOOG, H. (1976). The musical experience of the pre-school child. London: Schott. MORRIS, R. (1987). Composition with pitch-classes: A theory of compositional design. New Haven: Yale University Press. MUDD, S. A. (1963). Spatial stereotypes of four dimensions of pure tone. Journal of Experimental Psychology, 66, 347-352. NAKAMURA, A. (1987). The communication of dynamics between musicians and listeners through musical performance. Perception and Psychophysics, 41, 525-533. NAKAMURA, S., SADATO, N., OOHASHI, T., NISHINA, E., FUWAMOTO, Y., & YONEKURA, Y. (1999). Analysis of music–brain interaction with simultaneous measurement of regional 245 cerebral blood flow and electroencephalogram beta rhythm in human subjects. Neuroscience Letters, 275, 222-226. NARMOUR, E. (1990). The analysis and cognition of basic melodic structures: The implication-realization model. Chicago: University of Chicago Press. NARMOUR, E. (1992). The analysis and cognition of melodic complexity: The implication-realization model. Chicago: University of Chicago Press. NEUHOFF, J. G., & MCBEATH, M. K. (1996). The Doppler illusion: The influence of dynamic intensity change on perceived pitch. Journal of Experimental Psychology: Human Perception and Performance, 22, 970-985. NEUHOFF, J. G., MCBEATH, M. K., & WANZIE, W. C. (1999). Dynamic frequency change influences loudness perception: A central, analytic process. Journal of Experimental Psychology: Human Perception and Performance, 25, 1050-1059. NIELSEN, F. V. (1983). Oplevelse af musikalsk spænding [The Experience of Musical Tension]. Copenhagen: Akademisk Forlag. VAN NOORDEN, L. P. A. S. (1975). Temporal coherence in the perception of tone sequences. Doctoral dissertation, Eindhoven University of Technology, Holland. PAPOUSEK, M. (1996). Intuitive parenting: A hidden source of musical stimulation in infancy. In I. Deliège & J. Sloboda (Eds.), Musical beginnings: Origins and development of musical competence (pp. 88-112). Oxford: Oxford University Press. PENHUNE, V., ZATTORE, R. J., & EVANS, A. C. (1998). Cerebellar contributions to motor timing: A PET study of auditory and visual rhythm reproduction. Journal of Cognitive Neuroscience, 10, 752-765. PLATEL, H., PRICE, C., BARON, J. C., WISE, R., LAMBERT, J., FRACKOWIAK, R. S., LECHEVALIER, B., & EUSTACHE, F. (1997). The structural components of music perception: A functional anatomical study. Brain, 120, 229-243. PRATT, C. C. (1930). The spatial character of high and low tones. Journal of Experimental Psychology, 13, 278-285. PRENDERGAST, R. M. (1992). Film music: A neglected art. A critical study of music in films (2nd ed.). New York: Norton. REPP, B. H. (1992a). Diversity and commonality in musical performance. An analysis of timing microstructure in Schumann’s “Träumerei.” Journal of the Acoustical Society of America, 9, 2546-2568. REPP, B. H. (1992b). A constraint on the expressive timing of a melodic gesture: Evidence from performance and aesthetic judgment. Music Perception, 10, 221-242. REPP, B. H. (1993). Music as motion: A synopsis of Alexander Truslit’s “Gestaltung und Bewegung in der Musik” (1938). Psychology of Music, 21, 48-72. REPP, B. H. (1998). Musical motion in perception and performance. In D. A. Rosenbaum & C. E. Collyer (Eds.), 04.MUSIC.23_221-248.qxd 246 01/02/2006 12:26 Page 246 Z. Eitan and R. Y. Granot Timing of behavior: Neural, psychological, and computational perspectives (pp. 125-144). Cambridge, MA: MIT Press. REPP, B., WINDSOR, L., & DESAIN, P. (2002). Effects of tempo on the timing of simple musical rhythms. Music Perception, 19, 565-593. RINK, J. (1999). Translating musical meaning: The nineteenthcentury performer as narrator. In N. Cook & M. Everist (Eds.), Rethinking music (pp. 217-238). Oxford: Oxford University Press. ROFFLER, S. K., & BUTLER, R. A. (1968). Localization of tonal stimuli in the vertical plane. Journal of the Acoustical Society of America, 43, 1260-1265. SASLAW, J. (1996). Forces, containers, and paths: The role of body-derived image schemas in the conceptualization of music. Journal of Music Theory, 40, 217-243. SCHENKER, H. (1979). Free composition (E. Oster, Trans. & Ed.). New York: Longman. (Original work published 1935) SCRUTON, R. (1997). The aesthetics of music. Oxford: Clarendon Press. SHAPIRO, M. (1976). Asymmetry: An inquiry into the linguistic structure of poetry. Amsterdam: North Holland. SHOVE, P., & REPP, B. (1995). Musical motion and performance: Theoretical and empirical perspectives. In J. Rink (Ed.), The practice of performance: Studies in musical interpretation (pp. 55-83). Cambridge: Cambridge University Press. SPENCE, C., & DRIVER, J. (1997). Audiovisual links in exogenous overt spatial orienting. Perception & Psychophysics, 59, 1-22. SPITZER, M. (2004). Metaphor and musical thought. Chicago: Chicago University Press. STEIN, B. E., WALLACE, M. T., & MEREDITH, M. A. (1995). Neural mechanisms mediating attention and orientation to multisensory cues. In M. Gazzaniga (Ed.), The cognitive neurosciences (pp. 683-702). Cambridge, MA: MIT Press. STERN, D. N. (1985). The interpersonal world of the infant: A view from psychoanalysis and developmental psychology. New York: Basic Books. STEVENS, S. S. (1961). The psychophysics of the sensory function. In W. A. Rosenblict (Ed.), Sensory communication (pp. 1-33). New York: John Wiley and MIT Press. STEVENS, S. S. (1975). Psychophysics: Introduction to its perceptual, neural, and social prospects. New York: Wiley. STEVENS, S. S., & GUIRAO, M. (1963). Subjective scaling of length and area and the matching of length to loudness and brightness. Journal of Experimental Psychology, 66, 177-186. STEVENS, S. T., & ARIEH, Y. (2005). What you see is what you hear: The effect of auditory pitch on the detection of visual targets. Poster presented at the 76th annual meeting of the Eastern Psychological Society, Boston, MA. STEWART, L., WALSH, V., & FRITH, U. (2004). Reading music modifies spatial mapping in pianists. Perception &Psychophysics, 66, 183-195. SULLIVAN, J. W., & HOROWITZ, F. D. (1983). Infant intermodal perception and maternal multimodal stimulation: Implications for language development. In L. P. Lipsitt & C. K. Rovee-Collier (Eds.), Advances in infancy research (Vol. 2, pp. 183-239). Norwood, NJ: Ablex. SUNDBERG, J., & VERRILLO, V. (1980). On the anatomy of the retard: A study of timing in music. Journal of the Acoustical Society of America, 68, 772-779. TEKMAN, H. G. (1997). Interactions of perceived intensity, duration and pitch in pure tone sequences. Music Perception, 14, 281-294. TIMBLE, O. C. (1934). Localization of sound in the anteriorposterior and vertical dimensions of “auditory” space. British Journal of Psychology, 24, 320-334. TODD, N. P. M. (1985). A model of expressive timing in tonal music. Music Perception, 3, 33-58. TODD, N. P. M. (1992). The dynamics of dynamics: A model of musical expression. Journal of the Acoustical Society of America, 91, 3540-3550. TODD, N. P. M. (1994). The auditory “primal sketch”: A multiscale model of rhythmic grouping. Journal of New Music Research, 23, 25-70. TODD, N. P. M. (1995). The kinematics of musical expression. Journal of the Acoustical Society of America, 97, 1940-1949. TODD, N. P. M. (1999). Motion in music: A neurobiological perspective. Music Perception, 17, 115-126. TOLBERT, E. (1990). Women cry with words: Symbolization of affect in the Karelian lament. Yearbook for Traditional Music, 22, 80-105. VINES, B., WANDERLEY, M. M., NUZZO, R., LEVITIN, D., & KRUMHANSL, C. (2004). Performance gestures of musicians: What structural and emotional information do they convey? Proceedings of the 5th International Workshop on Gesture and Sign Language based Human-Computer Interaction. In the series “Lecture Notes in Artificial Intelligence.” Berlin: Springer. VOS, P. G., & TROOST, J. M. (1989). Ascending and descending melodic intervals: Statistical findings and their perceptual relevance. Music Perception, 6, 383-396. WAGNER, Y. S., WINNER, E., CICCHETTI, D., & GARDNER, H. (1981). “Metaphorical” mapping in human infants. Child Development, 52, 728-731. WALKER, B. N. (2000). Magnitude estimation of conceptual data dimensions for use in sonification. Doctoral dissertation, Rice University. WALKER, R. (1987). The effects of culture, environment, age, and musical training on choices of visual metaphors for sound. Perception and Psychophysics, 42, 491-502. WALLIN, N. L., MERKER, B., & BROWN, S. (Eds.), (2000). The origins of music. Cambridge, MA: MIT Press. WANDERLEY, M., & BATTIER, M. (Eds.), (2000). Trends in gestural control of music. Paris: Ircam. 04.MUSIC.23_221-248.qxd 01/02/2006 12:26 Page 247 How Music Moves: Musical Parameters and Listeners’ Images of Motion WEEKS, D. J., & PROCTOR, R. W. (1990). Salient-features coding in the translation between orthogonal stimulus and response dimensions. Journal of Experimental Psychology: General, 119, 355-366. WIDMANN, A., KUJALA, T., TERVANIEMI, M., KUJALA, A., & SCHROGER, E. (2004). From symbols to sounds: Visual symbolic information activates sound representations. Psychophysiology, 41, 709-715. WILLIAMON, R. A., & DAVIDSON, J. W. (2002). Exploring co-performer communication. Musicae Scientiae, 6, 1-17. WINDSOR, L., AARTS, R., DESAIN, P., HEIJINK, H., & TIMMERS, R. (2001). The timing of grace notes in skilled musical performance at different tempi: A preliminary case study. Psychology of Music, 29, 149-169. WÜHR, P., & MÜSSELER, J. (2002). Blindness to responsecompatible stimuli in the psychological refractory period paradigm. Visual Cognition, 9(4-5), 421-457. 247 ZATORRE, R. J. (2001). Do you see what I’m saying? Interactions between auditory and visual cortices in cochlear implant users. Neuron, 31, 13-14. ZATORRE, R. J., EVANS, A. C., & MEYER, E. J. (1994). Neural mechanisms underlying melodic perception and memory for pitch. Neuroscience, 14, 1908-1919. ZATORRE, R. J., MONDOR, T. A., & EVANS, A. C. (1999). Auditory attention to space and frequency activates similar cerebral systems. NeuroImage, 10, 544-554. ZBIKOWSKI, L. (1997). Conceptual models and cross-domain mapping: New perspectives on theories of music and hierarchy. Journal of Music Theory, 41, 193-225. ZBIKOWSKI, L. (1998). Metaphor and music theory. Music Theory Online, 4. Retrieved from http://societymusictheory.org/mto/issues/mto.98.4.1/mto.98. 4.1.zbikowski.html ZBIKOWSKI, L. (2002). Conceptualizing music: Cognitive structure, theory, and analysis. New York: Oxford University Press. 04.MUSIC.23_221-248.qxd 01/02/2006 12:26 Page 248 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. View publication stats