research-article

The Transfer of Learning as HCI Similarity: Towards an Objective Assessment of the Sensory-Motor Basis of Naturalness

Authors:

François Bérard,

Amélie Rochet-CapellanAuthors Info & Claims

CHI '15: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems

Pages 1315 - 1324

https://doi.org/10.1145/2702123.2702359

Published: 18 April 2015 Publication History

Abstract

Human-computer interaction should be natural. However, the notion of natural is questioned due to a lack of theoretical background and methods to objectively measure the naturalness of a HCI. A frequently cited aspect of natural HCIs is their ability to benefit from knowledge and skills that users develop in their interaction with the real (non-digital) world. Among these skills, sensory-motor abilities are essential to operate many HCIs. This suggests that the transfer of these abilities between physical and digital interactions could be used as an experimental tool to assess the sensory-motor similarity between interactions, and could be considered as an objective measurement of the sensory-motor grounding of naturalness. In this framework, we introduce a new experimental paradigm inspired by motor learning research to assess sensory-motor similarity, as revealed by the transfer of learning. We tested this paradigm in an empirical study to question the naturalness of three HCIs: direct-touch, mouse pointing and absolute indirect-touch. The study revealed how skill learning transfers from these three digital interactions towards an equivalent physical interaction. We observed strong transfer of skill between direct-touch and physical interaction, but no transfer from the other two interactions. This work provides a first objective assessment of the sensory-motor basis of direct-touch naturalness, and a new empirical path to question HCI similarity and naturalness.

Supplementary Material

suppl.mov (pn1188-file3.mp4)

Supplemental video

Download
26.83 MB

References

[1]

Accot, J., and Zhai, S. More than dotting the i's -- foundations for crossing-based interfaces. In ACM Conference on Human Factors in Computing Systems (CHI), ACM (2002), 73--80.

Digital Library

[2]

Adams, J. A. Historical review and appraisal of research on the learning, retention, and transfer of human motor skills. Psychological Bulletin 101, 1 (1987), 41.

[3]

Beaudouin-Lafon, M. Designing interaction, not interfaces. In Proceedings of the working conference on Advanced visual interfaces, ACM (2004), 15--22.

Digital Library

[4]

Belda-Lois, J.-M., de Rosario, H., Pons, R., Poveda, R., Morón, A., Porcar, R., Gómez, A., et al. Can human movement analysis contribute to usability understanding? Human movement science 29, 4 (2010), 529--541.

[5]

Card, S., Moran, T. P., and Newell, A. The Psychology of Human-Computer Interaction. Lawrence Erlbaum Associates, 1983.

Digital Library

[6]

Clower, D. M., and Boussaoud, D. Selective use of perceptual recalibration versus visuomotor skill acquisition. Journal of Neurophysiology 84, 5 (2000), 2703--2708.

[7]

Fallman, D. The new good: exploring the potential of philosophy of technology to contribute to human-computer interaction. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM (2011), 1051--1060.

Digital Library

[8]

Fitts, P. M. The information capacity of the human motor system in controlling the amplitude of movement. Journal of Experimental Psychology 47, 6 (1954), 381--391.

[9]

Gandolfo, F., Mussa-Ivaldi, F., and Bizzi, E. Motor learning by field approximation. Proceedings of the National Academy of Sciences 93, 9 (1996), 3843--3846.

[10]

Ghilardi, M. F., Moisello, C., Silvestri, G., Ghez, C., and Krakauer, J. W. Learning of a sequential motor skill comprises explicit and implicit components that consolidate differently. J Neurophysiol 101, 5 (May 2009), 2218--2229.

[11]

Gillies, M., and Kleinsmith, A. Non-representational interaction design. In Contemporary Sensorimotor Theory. Springer, 2014, 201--208.

[12]

Holden, M. K. Virtual environments for motor rehabilitation: review. Cyberpsychology & behavior 8, 3 (2005), 187--211.

[13]

Houde, J. F., and Jordan, M. I. Sensorimotor adaptation in speech production. Science 279, 5354 (1998), 1213--1216.

[14]

Jacob, R. J., Girouard, A., Hirshfield, L. M., Horn, M. S., Shaer, O., Solovey, E. T., and Zigelbaum, J. Reality-based interaction: a framework for post-wimp interfaces. In Proceedings of the SIGCHI conference on Human factors in computing systems, ACM (2008), 201--210.

Digital Library

[15]

Jetter, H.-C., Leifert, S., Gerken, J., Schubert, S., and Reiterer, H. Does (multi-)touch aid users' spatial memory and navigation in 'panning' and in 'zooming & panning' uis? In Proceedings of AVI, ACM (2012), 83--90.

Digital Library

[16]

Jetter, H.-C., Reiterer, H., and Geyer, F. Blended interaction: understanding natural human-computer interaction in post-wimp interactive spaces. Personal and Ubiquitous Computing 18, 5 (2014), 1139--1158.

Digital Library

[17]

Kirsh, D. Embodied cognition and the magical future of interaction design. ACM Transactions on Computer-Human Interaction (TOCHI) 20, 1 (2013), 3.

Digital Library

[18]

Krakauer, J. W., Ghez, C., and Ghilardi, M. F. Adaptation to visuomotor transformations: consolidation, interference, and forgetting. The Journal of Neuroscience 25, 2 (2005), 473--478.

[19]

Krakauer, J. W., and Mazzoni, P. Human sensorimotor learning: adaptation, skill, and beyond. Current opinion in neurobiology 21, 4 (2011), 636--644.

[20]

Krakauer, J. W., Mazzoni, P., Ghazizadeh, A., Ravindran, R., and Shadmehr, R. Generalization of motor learning depends on the history of prior action. PLoS biology 4, 10 (2006), e316.

[21]

Mattar, A. A., and Ostry, D. J. Modifiability of generalization in dynamics learning. Journal of neurophysiology 98, 6 (2007), 3321--3329.

[22]

Norman, D. A. The way i see it: Natural user interfaces are not natural. interactions 17, 3 (2010), 6--10.

Digital Library

[23]

O'hara, K., Harper, R., Mentis, H., Sellen, A., and Taylor, A. On the naturalness of touchless: putting the "interaction" back into nui. ACM Transactions on Computer-Human Interaction (TOCHI) 20, 1 (2013), 5.

Digital Library

[24]

Poddar, I., Sethi, Y., Ozyildiz, E., and Sharma, R. Toward natural gesture/speech hci: A case study of weather narration. In Proc. Workshop on Perceptual User Interfaces (PUI), Citeseer (1998).

[25]

Shadmehr, R. Generalization as a behavioral window to the neural mechanisms of learning internal models. Human movement science 23, 5 (2004), 543--568.

[26]

Soukoreff, R. W., and MacKenzie, I. S. Towards a standard for pointing device evaluation, perspectives on 27 years of fitts' law research in hci. International Journal of Human-Computer Studies 61, 6 (2004), 751--789.

Digital Library

[27]

Tan, D. S., Pausch, R., Stefanucci, J. K., and Proffitt, D. R. Kinesthetic cues aid spatial memory. In CHI '02 Extended Abstracts on Human Factors in Computing Systems, ACM (2002), 806--807.

Digital Library

[28]

Tremblay, S., Houle, G., and Ostry, D. J. Specificity of speech motor learning. The Journal of Neuroscience 28, 10 (2008), 2426--2434.

[29]

Van Dam, A. Post-wimp user interfaces. Communications of the ACM 40, 2 (1997), 63--67.

Digital Library

[30]

Wei, K., Yan, X., Kong, G., Yin, C., Zhang, F., Wang, Q., and Kording, K. P. Computer use changes generalization of movement learning. Current Biology 24, 1 (2014), 82--85.

[31]

Wigdor, D., and Wixon, D. Brave NUI world: designing natural user interfaces for touch and gesture. Elsevier, 2011.

Digital Library

[32]

Wobbrock, J. O., Aung, H. H., Rothrock, B., and Myers, B. A. Maximizing the guessability of symbolic input. In ACM Conference on Human Factors in Computing Systems (CHI) Extended Abstracts, ACM (2005), 1869--1872.

Digital Library

[33]

Wobbrock, J. O., Morris, M. R., and Wilson, A. D. User-defined gestures for surface computing. In ACM Conference on Human Factors in Computing Systems (CHI), ACM (2009), 1083--1092.

Digital Library

[34]

Wolpert, D. M., Ghahramani, Z., and Flanagan, J. R. Perspectives and problems in motor learning. Trends in cognitive sciences 5, 11 (2001), 487--494.

[35]

Zhu, F. F., Poolton, J. M., Maxwell, J. P., Fan, J. K. M., Leung, G. K. K., and Masters, R. S. W. Refining the continuous tracking paradigm to investigate implicit motor learning. Exp Psychol 61, 3 (2014), 196--204.

Cited By

Renom MCaramiaux BBeaudouin-Lafon M(2022)Exploring Technical Reasoning in Digital Tool UseProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3501877(1-17)Online publication date: 29-Apr-2022
https://dl.acm.org/doi/10.1145/3491102.3501877
Louis TTroccaz JRochet-Capellan AHoyek NBérard F(2020)When High Fidelity MattersProceedings of the 2020 International Conference on Advanced Visual Interfaces10.1145/3399715.3399815(1-9)Online publication date: 28-Sep-2020
https://dl.acm.org/doi/10.1145/3399715.3399815
Raissi RDimara EBerry JGray WBailly GBernhaupt RMueller FVerweij DAndres JMcGrenere JCockburn AAvellino IGoguey ABjørn PZhao SSamson BKocielnik R(2020)Retroactive Transfer Phenomena in Alternating User InterfacesProceedings of the 2020 CHI Conference on Human Factors in Computing Systems10.1145/3313831.3376538(1-14)Online publication date: 21-Apr-2020
https://dl.acm.org/doi/10.1145/3313831.3376538
Show More Cited By

Index Terms

The Transfer of Learning as HCI Similarity: Towards an Objective Assessment of the Sensory-Motor Basis of Naturalness
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. HCI design and evaluation methods
    2. HCI theory, concepts and models
  2. Interaction design
    1. Interaction design theory, concepts and paradigms

Recommendations

Touchless circular menus: toward an intuitive UI for touchless interactions with large displays
AVI '14: Proceedings of the 2014 International Working Conference on Advanced Visual Interfaces

Researchers are exploring touchless interactions in diverse usage contexts. These include interacting with public displays, where mouse and keyboards are inconvenient, activating kitchen devices without touching them with dirty hands, or supporting ...
Investigating the interaction of children through NUI in e-learning applications
IHC '17: Proceedings of the XVI Brazilian Symposium on Human Factors in Computing Systems

Although the use of natural user interfaces (NUI) has been growing, there is still a discussion of what is "natural" to a specific audience. This article presents an investigation on how children use and understand hands' natural movements (gestures) ...
Involving users in the gestural language definition process for the NInA framework
IHC '13: Proceedings of the 12th Brazilian Symposium on Human Factors in Computing Systems

Along the past years, many interaction devices had been introduced to support the interaction between humans and computers, as the digital pen, touch sensitive devices and, more recently, gesture recognition devices. The last one allows a human-computer ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '15: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems

April 2015

4290 pages

ISBN:9781450331456

DOI:10.1145/2702123

General Chairs:
Bo Begole
Huawei, USA
,
Jinwoo Kim
Yonsei University, Korea
,
Program Chairs:
Kori Inkpen
Microsoft Research, USA
,
Woontack Woo
KAIST, Korea

Copyright © 2015 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 April 2015

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

ANR

Conference

CHI '15

Sponsor:

SIGCHI

CHI '15: CHI Conference on Human Factors in Computing Systems

April 18 - 23, 2015

Seoul, Republic of Korea

Acceptance Rates

CHI '15 Paper Acceptance Rate 486 of 2,120 submissions, 23%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

14
Total Citations
View Citations
392
Total Downloads

Downloads (Last 12 months)31
Downloads (Last 6 weeks)4

Reflects downloads up to 25 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Renom MCaramiaux BBeaudouin-Lafon M(2022)Exploring Technical Reasoning in Digital Tool UseProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3501877(1-17)Online publication date: 29-Apr-2022
https://dl.acm.org/doi/10.1145/3491102.3501877
Louis TTroccaz JRochet-Capellan AHoyek NBérard F(2020)When High Fidelity MattersProceedings of the 2020 International Conference on Advanced Visual Interfaces10.1145/3399715.3399815(1-9)Online publication date: 28-Sep-2020
https://dl.acm.org/doi/10.1145/3399715.3399815
Raissi RDimara EBerry JGray WBailly GBernhaupt RMueller FVerweij DAndres JMcGrenere JCockburn AAvellino IGoguey ABjørn PZhao SSamson BKocielnik R(2020)Retroactive Transfer Phenomena in Alternating User InterfacesProceedings of the 2020 CHI Conference on Human Factors in Computing Systems10.1145/3313831.3376538(1-14)Online publication date: 21-Apr-2020
https://dl.acm.org/doi/10.1145/3313831.3376538
Louis TTroccaz JRochet-Capellan ABérard FLee BLee GScott STory MKim J(2019)Is it Real? Measuring the Effect of Resolution, Latency, Frame rate and Jitter on the Presence of Virtual EntitiesProceedings of the 2019 ACM International Conference on Interactive Surfaces and Spaces10.1145/3343055.3359710(5-16)Online publication date: 10-Nov-2019
https://dl.acm.org/doi/10.1145/3343055.3359710
Hornbæk KMottelson AKnibbe JVogel D(2019)What Do We Mean by “Interaction”? An Analysis of 35 Years of CHIACM Transactions on Computer-Human Interaction10.1145/332528526:4(1-30)Online publication date: 1-Jul-2019
https://dl.acm.org/doi/10.1145/3325285
Morreale FArmitage JMcPherson A(2018)Effect of Instrument Structure Alterations on Violin PerformanceFrontiers in Psychology10.3389/fpsyg.2018.024369Online publication date: 6-Dec-2018
https://doi.org/10.3389/fpsyg.2018.02436
Bergstrom-Lehtovirta JCoyle DKnibbe JHornbæk KMandryk RHancock MPerry MCox A(2018)I Really did ThatProceedings of the 2018 CHI Conference on Human Factors in Computing Systems10.1145/3173574.3173952(1-8)Online publication date: 21-Apr-2018
https://dl.acm.org/doi/10.1145/3173574.3173952
Cattan EPerrier PBérard FGerber SRochet-Capellan A(2018)Adaptation to visual feedback delays on touchscreens with hand visionExperimental Brain Research10.1007/s00221-018-5368-2236:12(3191-3201)Online publication date: 6-Sep-2018
https://doi.org/10.1007/s00221-018-5368-2
Louis TBerard FSubramanian SSteimle JDachselt RMartinez Plasencia DGrossman T(2017)Superiority of a Handheld Perspective-Coupled Display in Isomorphic Docking PerformancesProceedings of the 2017 ACM International Conference on Interactive Surfaces and Spaces10.1145/3132272.3134124(72-81)Online publication date: 17-Oct-2017
https://dl.acm.org/doi/10.1145/3132272.3134124
Cattan ERochet-Capellan APerrier PBérard FMark GFussell SLampe Cschraefel mHourcade JAppert CWigdor D(2017)Does Practice Make Perfect?Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems10.1145/3025453.3025585(5619-5629)Online publication date: 2-May-2017
https://dl.acm.org/doi/10.1145/3025453.3025585
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents