Abstract
This paper presents a novel web platform designed to investigate the relationship between sounds, colors, and emotions, with the overarching goal of enhancing the sensory experience of impaired individuals in visual art museums. Taking advantage of the principles of sensory substitution, the project aims to bridge the gap between auditory and visual perception, allowing individuals with visual or cognitive impairments to engage with visual art through alternative sensory modalities. The platform’s architecture is centered around the delivery of short sound stimuli to participants, who then provide feedback on the associations they perceive between these auditory cues and both colors and emotional dimensions. The collected data will be analyzed to discern patterns and correlations, shedding light on how auditory stimuli can be used to evoke visual and emotional responses. This paper outlines the technical and methodological aspects of the web platform, including its design, development, and implementation. It discusses the selection of sound stimuli and the integration of user-friendly interfaces to ensure a seamless experience for participants. Preliminary results from volunteer tests are briefly presented, highlighting intriguing findings regarding the associations between sounds, colors, and emotions.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
From a formal point of view, white and pink noise are very likely to be additional examples of Electrophone-generated timbres.
- 2.
World Wide Web Consortium, https://www.w3.org/.
- 3.
References
Abboud, S., Hanassy, S., Levy-Tzedek, S., Maidenbaum, S., Amedi, A.: Eyemusic: introducing a visual colorful experience for the blind using auditory sensory substitution. Restor. Neurol. Neurosci. 32(2), 247–257 (2014)
Berlin, B., Kay, P.: Basic Color Terms: Their Universality and Evolution. University of California Press (1969)
Bologna, G., Gomez, J.D., Pun, T.: Vision substitution experiments with see color. In: Ferrández Vicente, J.M., Álvarez Sánchez, J.R., de la Paz López, F., Toledo Moreo, F.J. (eds.) IWINAC 2013. LNCS, vol. 7930, pp. 83–93. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-38637-4_9
Bradley, M.M., Lang, P.J.: Measuring emotion: the self-assessment manikin and the semantic differential. J. Behav. Ther. Exp. Psychiatry 25(1), 49–59 (1994)
Castel, L.B.: L’optique des couleurs: fondée sur les simples observations, & tournée sur-tout à la pratique de la peinture, de la teinture & des autres arts coloristes. chez Briasson (1740)
Cavaco, S., Henriques, J.T., Mengucci, M., Correia, N., Medeiros, F.: Color sonification for the visually impaired. Procedia Technol. 9, 1048–1057 (2013)
Collopy, F.: Rhythmic light, hue to pitch. https://rhythmiclight.com/visual-musiccorrespondences/hue-to-pitch/ (2004). Accessed 16 May 2023
Cuddy, L.L.: The color of melody. Music. Percept. 2(3), 345–360 (1985). https://doi.org/10.2307/40285303
Day, S.: Some demographic and socio-cultural aspects of synesthesia. In: Robertson, L., N. Sagiv, N. (eds.) Synesthesia: Perspectives from Cognitive Neuroscience, chap. 2, pp. 11–33. Oxford University Press, Oxford (2005)
Day, S.A.: Synesthesia and Synesthetes. Independently published (2022)
Dini, S., Ludovico, L.A., Mascetti, S., Valero Gisbert, M.J.: Translating color: sonification as a method of sensory substitution within the museum. In: 20th International Web for All Conference (W4A ’23), April 30-May 1, 2023, Austin, TX, USA, pp. 162–163. ACM (2023). https://doi.org/10.1145/3587281.3587706
Franssen, M.: The ocular harpsichord of Louis-Bertrand Castel. Tractrix 3(1991), 15–77 (1991)
Gepner, C.: Le Père Castel et le clavecin oculaire: carrefour de ĺesthétique et des savoirs dans la première moitié du XVIIIe siècle. Ph.D. thesis, Paris 3 (1994)
Hawkins, V.: Music-color synesthesia: a historical and scientific overview. Aisthesis: Honors Student J. 13(1) (2022)
Hermann, T., Hunt, A., Neuhoff, J.G., et al.: The Sonification Handbook, vol. 1. Logos Verlag Berlin (2011)
Lee, D.: Hornbostel-Sachs classification of musical instruments. Knowl. Organ. 47(1), 72–91 (2019)
Levkowitz, H.: Color Theory and Modeling for Computer Graphics, Visualization, and Multimedia Applications. Springer (1997). https://doi.org/10.1007/b102382
Maidenbaum, S., Abboud, S., Amedi, A.: Sensory substitution: closing the gap between basic research and widespread practical visual rehabilitation. Neurosci. Biobehav. Rev. 41, 3–15 (2014)
Moritz, W.: The dream of color music, and machines that made it possible. Animation World Mag. 2(1) (1997)
Newton, I.: Opticks, or, a treatise of the reflections, refractions, inflections & colours of light. Courier Corporation (1952)
Niccolai, V., Jennes, J., Stoerig, P., Van Leeuwen, T.M.: Modality and variability of synesthetic experience. Am. J. Psychol. 125(1), 81–94 (2012). www.jstor.org/stable/10.5406/amerjpsyc.125.1.0081
Peacock, K.: Instruments to perform color-music: two centuries of technological experimentation. Leonardo 21(4), 397–406 (1988). https://doi.org/10.2307/1578702
Plutchik, R., Kellerman, H.: Emotion, theory, research, and experience: theory, research and experience. Academic Press (1980)
Presti, G., et al.: WatchOut: obstacle sonification for people with visual impairment or blindness. In: Proceedings of the 21th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS’19), pp. 402–413. ACM (2019). https://doi.org/10.1145/3308561.3353779
Proulx, M.J., Brown, D.J., Pasqualotto, A., Meijer, P.: Multisensory perceptual learning and sensory substitution. Neurosci. Biobehav. Rev. 41, 16–25 (2014)
Pun, T., Deville, B., Bologna, G.: Sonification of colour and depth in a mobility aid for blind people. In: 16th International Conference on Auditory Display (ICAD-2010), pp. 9–13. Georgia Institute of Technology (2010)
Rector, K., Salmon, K., Thornton, D., Joshi, N., Morris, M.R.: Eyes-free art: exploring proxemic audio interfaces for blind and low vision art engagement. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 1(3) (2017). https://doi.org/10.1145/3130958
Saglietti, B.: Dal clavecin oculaire di Louis Bertrand Castel al clavier à lumières di Alexandr Skrjabin. In: Metamorfosi dei Lumi 6. Belle lettere e scienza, pp. 187–205. Accademia Unviversity Press (2012)
Simner, J., et al.: Synaesthesia: the prevalence of atypical cross-modal experiences. Perception 35(8), 1024–1033 (2006)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Dini, S., Ludovico, L.A., Rizzi, A., Sarti, B., Valero Gisbert, M.J. (2023). A Web Platform to Investigate the Relationship Between Sounds, Colors and Emotions. In: da Silva, H.P., Cipresso, P. (eds) Computer-Human Interaction Research and Applications. CHIRA 2023. Communications in Computer and Information Science, vol 1997. Springer, Cham. https://doi.org/10.1007/978-3-031-49368-3_15
Download citation
DOI: https://doi.org/10.1007/978-3-031-49368-3_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-49367-6
Online ISBN: 978-3-031-49368-3
eBook Packages: Computer ScienceComputer Science (R0)