Abstract
This research develops a novel way of rethinking cultural and social behavior using computationally augmented artifacts. These ‘instruments’ provide various types of auditory feedback when manipulated by certain actions within social contexts, such as a bar or dining space. They foster affective social engagement through the habitual and explorative actions that they afford in everyday contexts, and their resulting auditory feedback. The goal is not only to observe how social interactions are affected by the manipulation of augmented artifacts, but also to observe how the sounds and manipulations affect psycho-sociological [1] changes towards more collaborative social relations during the processes of participatory sense-making [2]. In this paper, we present: a) a study of dynamic social interaction and how we instrumented tangible artifacts to reflect and induce engagement, b) a literature review that provides background for our design methodology, c) ‘vocal prototyping’–a responsive media technique for developing action-sonic mappings, d) our experimental prototype based on this design methodology.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
In Leigh Star and Griesemer’s formulation: “Boundary objects are objects which are both plastics enough to adapt to local needs and constraints of the several parties employing them, yet robust enough to maintain a common identity across sites. They are weakly structured in common use and become strongly structured in individual-site use. They have different meanings in different social worlds, but their structure is common enough to more than one world to make them recognizable, a means of translation. The creation and management of boundary objects is key in developing and maintaining coherence across intersecting social worlds.” [34, p 393].
- 2.
We thank and acknowledge Shomit Barua for this formulation in terms of conviviality [28].
- 3.
Where we use “sense” as a verb, we explicitly take it to mean the way a human (or a living organism) takes stock of its relation to its ambient. We take a Deleuzian interpretation of the noun sense as the bundle of “processes of [differentiation], transformation, resistance or appropriation inflicted on a thing.” [24, p. 10]
- 4.
- 5.
Topological Media Lab, 2001- present, http://topologicalmedialab.net
- 6.
Concurrent or responsive coordination, and synchronization are first approximations to the richer phenomena of coordination via rhythm as generalized, multidimensional or aperiodic, temporal form, in work by Garrett Johnson and the last author [31], and with Adrian Freed, Todd Ingalls, Julian Stein, Gabrielle Isaacs at Synthesis.
- 7.
Adafruit Feather M0. https://www.adafruit.com/product/2772
- 8.
Adafruit BNO055 Absolute Orientation Sensor. https://learn.adafruit.com/adafruit-bno055-absolute-orientation-sensor
References
Lemaitre, G., et al.: Feelings elicited by auditory feedback from a computationally augmented artifact: the flops. IEEE Trans. Affect. Comput. 3(3), 335–348 (2012). https://doi.org/10.1109/t-affc
De Jaegher, H., Di Paolo, E.: Participatory Sense-Making. Phenomenol. Cogn. Sci. 6(4), 485–507 (2007). https://doi.org/10.1007/s11097-007-9076-9
De Jaegher, H., Di Paolo, E., Gallagher, S.: Can social interaction constitute social cognition? Trends Cogn. Sci. 14(10): 441–447 (2010). https://doi.org/10.1016/j.tocs
Hayes, L.: Beyond Skill Acquisition: Improvisation, Interdisciplinarity, and Enactive Music Cognition. Contemporary Music Review. Taylor & Francis, London (2019)
Coccia, E.: Reversing the New Global Monasticism. Fall Semester, 21, Apr. https://fallsemester.org/2020-1/2020/4/17/emanuele-coccia-escaping-the-global-monasticism (2020)
Fox, N.J.: Boundary Objects: Social Meanings and the Success of New Technologies. Sociology 45(1), 70–85 (2011)
Lyu, Y., Hayes, L.: Exploring social coordination through computationally augmented artifacts using auditory feedback. In: 26th International Symposium on Electronic, Montreal, Canada (accepted, 2020)
Tanaka, S.: Intercorporeality and Aida: developing an interaction theory of social cognition. Theory Psychol. 27(3), 337–353 (2017)
Gendlin, E.T.: Experiencing and the Creation of Meaning: a philosophical and psychological approach to the subjective. Free Press of Glencoe, New York (1962)
Gendlin, E.T.: Thinking beyond patterns: Body, language and situations. Focusing Institute, Spring Valley (1991)
Gill, S.P.: Tacit engagement: beyond interaction. AI & Soc. 34(1), 163 (2016). https://doi.org/10.1007/s00146-016-0681-4
Sha, X. W.: The Atelier-Lab as a Transversal Machine. Revue rancaise d’etudes americaines, pp. 62–78 (2011)
Sha, X.W., Freed, A., Navab, N.: Sound design as human matter interaction. Extended Abstracts on Human Factors in Computing Systems, April 2013, pp. 2009–2018. https://doi.org/10.1145/2468356.2468718
Rafaeli, A., Vilnai-Yavetz, I.: Instrumentality, aesthetics and symbolism of physical artifacts as triggers of emotion. Theoret. Issues Ergon. Sci. 5(1), 91–112 (2004). https://doi.org/10.1080/1463922031000086735
Johnston, T., Van Reekum, C., Hird, K., Kirsner, K., Scherer, K.R.: Affective speech elicited with a computer game. Emotion 5(4), 513–518 (2005)
Van Reekum, C., Johnston, T., Banse, R., Etter, A., Wehrle, T., Scherer, K.: Psychophysiological responses to appraisal dimensions in a computer game. Cogn. Emot. 18(5), 663–668 (2004)
Kaiser, S., Edwards, T.: Multimodal emotions measurement in an interactive computer game: a pilot-study. In: Proceedings of the VIIIth Conference of the International Society for Research on Emotions, vol. 13, no. 1, pp. 19–124 (1999)
Ravaja, N., Timo, S., Jani, L., Kari, K, Mikko.: The psychophysiology of video gaming: phase emotional response to game events. In Proceedings of the Digital Game Research Association (DiGra)conference: changing views -World in play (2005)
Ravaja, N., Turpeinen, M., Saari, T., Puttonen, S., Jarvinen, L.: The psychophysiology of games bond: phasic emotional response to violent video games. Emotion 8(1), 114–120 (2008)
Franinovic, K.: The Flops: Negotiating Between Habitual and Explorative Gestures. New Interfaces for Musical Expression (2011)
Polotti, P., Monache, S.D., Papetti, S., Rocchesso, D.: Gamelunch: forging a dining experience through sound. In: Proceeding of the Twenty- Sixth Annual CHI Conference Extended Abstracts on Human Factors in Computing Systems- CHI (2008). https://doi.org/10.1145/1358628.1358670
Shockley, K., Richardson, D.C., Dale, R.: Conversation and coordinative structures. Topics Cogn. Sci. 1(2), 305–319 (2009)
Richardson, M.J., et al.: Rocking together: dynamics of intentional and unintentional interpersonal coordination. Hum. Movement Sci. 26(6), 867–891 (2007)
Voss, D.: Deleuze’s Rethinking of the Notion of Sense. Deleuze Stud. 7(1), 1–25 (2003)
Gilbert, S.: On the Mode of Existence of Technical Objects. Translated by Cecile Malaspina and John Rogove, Minnesota (2015)
Kean, M.L.: Markedness. In International encyclopedia of linguistics, v. 2. Edited by W. Bright, pp. 390–391. Oxford University Press, Oxford (1992)
Fantauzza, J., Berzowska, J., Dow, S., Iachello, G., Sha, X.W.: Greeting Dynamics Using Expressive Softwear. Ubicomp (2003)
Barua, S., Lyu, Y.: Conviviality Cafe. Connectivity_Café (2020). http://synthesiscenter.net/projects/connectivity-cafe/(2019-2020)
Gaver, B., Dunne, T., Pacenti, E.: Cultural Probes. Interactions, Jan-Feb 21–29 (1999)
Graham, C., Rouncefield, M.: Probes and participation. In: Proceedings of the Tenth Anniversary Conference on Participatory Design, pp. 194–197(2008)
Sha, X.W., Johnson, G.L.: Rhythm and textural temporality: experience without subject and duration as effect. In: Crespi, P., Manghani, S. (eds.) Rhythm and Critique: Technics, Modalities, Practices,. Edinburgh (2020)
Rietveld, E., Kiverstein, J.: A rich landscape of affordances. Ecol. Psychol. 26(4), 325–352 (2014)
Hayes, L.: Enacting Musical Worlds: Common Approaches to using NIMEs within both Performance and Person-Centred Arts Practices. In: Proceedings of the International Conference on New Interfaces for Musical Expression, pp. 299–302. Baton Rouge, USA (2015)
Leigh Star, S.: The structure of Ill-structured solutions: boundary objects and heterogeneous distributed problem solving. In: Hubs, M., Gasser, L. (eds.) Readings in Distributed Artificial Intelligence 3. Morgan Kaufmann, Menlo Park (1989)
Krzyzaniak, M., Anirudh, R., Venkataraman, V., Turaga, P., Sha, X.W.: Towards realtime measurement of connectedness in human movement. Movement and Computing (2015)
Sha, X.W.: Poiesis and Enchantment in Topological Matter. MIT Press, Hoboken (2013)
Acknowledgments
For critical and technical support, we thank our research colleagues at the Synthesis Center and in the School of Arts, Media and Engineering at ASU: Shomit Barua, Assegid Kidane, Seth Thorn, Tejaswi Gowda, Connor Rawls, Todd Ingalls, and Peter Weisman.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Lyu, Y., Mechtley, B., Hayes, L., Sha, X.W. (2020). Tableware: Social Coordination Through Computationally Augmented Everyday Objects Using Auditory Feedback. In: Stephanidis, C., Duffy, V.G., Streitz, N., Konomi, S., Krömker, H. (eds) HCI International 2020 – Late Breaking Papers: Digital Human Modeling and Ergonomics, Mobility and Intelligent Environments. HCII 2020. Lecture Notes in Computer Science(), vol 12429. Springer, Cham. https://doi.org/10.1007/978-3-030-59987-4_24
Download citation
DOI: https://doi.org/10.1007/978-3-030-59987-4_24
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-59986-7
Online ISBN: 978-3-030-59987-4
eBook Packages: Computer ScienceComputer Science (R0)