research-article

Proxemic Cursor Interactions for Touchless Widget Control

Authors:

Euan FreemanAuthors Info & Claims

SUI '23: Proceedings of the 2023 ACM Symposium on Spatial User Interaction

Article No.: 21, Pages 1 - 12

https://doi.org/10.1145/3607822.3614525

Published: 13 October 2023 Publication History

Abstract

Touchless gesture interfaces often use cursor-based interactions, where widgets are targeted by a movable cursor and activated with a mid-air gesture (e.g., push or Pinch). Continuous interactions like slider manipulation can be challenging in mid-air because users need to precisely target widgets and then maintain an ‘activated’ state whilst moving the cursor. We investigated proxemic cursor interactions as a novel alternative, where cursor proximity allows users to acquire and keep control of user interface widgets without precisely targeting them. Users took advantage of proxemic targeting, though gravitated towards widgets when negotiating the boundaries between multiple elements. This allowed users to gain control more quickly than with non-proxemic behaviour, and made it easier to move between user interface elements. We find that proxemic cursor interactions can improve the usability of touchless user interfaces, especially for slider interactions, paving the way to more comfortable and efficient use of touchless displays.

Supplemental Material

MP4 File

Full-resolution scatterplots, Concept Video

Download
22.83 MB

PDF File

Full-resolution scatterplots, Concept Video

Download
1.67 MB

References

[1]

Florian Alt, Andreas Bulling, Gino Gravanis, and Daniel Buschek. 2015. GravitySpot: Guiding Users in Front of Public Displays Using On-Screen Visual Cues. In Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology (Charlotte, NC, USA) (UIST ’15). Association for Computing Machinery, New York, NY, USA, 47–56. https://doi.org/10.1145/2807442.2807490

Digital Library

[2]

Myroslav Bachynskyi and Jörg Müller. 2020. Dynamics of Aimed Mid-Air Movements. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3313831.3376194

Digital Library

[3]

Marc Baloup, Thomas Pietrzak, and Géry Casiez. 2019. RayCursor: A 3D Pointing Facilitation Technique Based on Raycasting. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3290605.3300331

Digital Library

[4]

Ishan Chatterjee, Robert Xiao, and Chris Harrison. 2015. Gaze+Gesture: Expressive, Precise and Targeted Free-Space Interactions. In Proceedings of the 2015 ACM on International Conference on Multimodal Interaction (Seattle, Washington, USA) (ICMI ’15). Association for Computing Machinery, New York, NY, USA, 131–138. https://doi.org/10.1145/2818346.2820752

Digital Library

[5]

Christopher Clarke and Hans Gellersen. 2017. MatchPoint: Spontaneous Spatial Coupling of Body Movement for Touchless Pointing. In Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology (Québec City, QC, Canada) (UIST ’17). Association for Computing Machinery, New York, NY, USA, 179–192. https://doi.org/10.1145/3126594.3126626

Digital Library

[6]

Sven Coppers, Kris Luyten, Davy Vanacken, David Navarre, Philippe Palanque, and Christine Gris. 2019. Fortunettes: Feedforward about the Future State of GUI Widgets. Proc. ACM Hum.-Comput. Interact. 3, EICS, Article 20 (jun 2019), 20 pages. https://doi.org/10.1145/3331162

Digital Library

[7]

Patricia Ivette Cornelio Martinez, Silvana De Pirro, Chi Thanh Vi, and Sriram Subramanian. 2017. Agency in Mid-air Interfaces. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems(CHI ’17). Association for Computing Machinery, New York, NY, USA, 2426–2439. https://doi.org/10.1145/3025453.3025457

Digital Library

[8]

Sean Cronin, Euan Freeman, and Gavin Doherty. 2022. Investigating Clutching Interactions for Touchless Medical Imaging Systems. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems(CHI ’22). Association for Computing Machinery, New York, NY, USA, 14 pages. https://doi.org/10.1145/3491102.3517512

Digital Library

[9]

Ashley Dover, G. Michael Poor, Darren Guinness, and Alvin Jude. 2016. Improving Gestural Interaction With Augmented Cursors. In Proceedings of the 2016 Symposium on Spatial User Interaction (Tokyo, Japan) (SUI ’16). Association for Computing Machinery, New York, NY, USA, 135–138. https://doi.org/10.1145/2983310.2985765

Digital Library

[10]

Lisa A. Elkin, Matthew Kay, James J. Higgins, and Jacob O. Wobbrock. 2021. An Aligned Rank Transform Procedure for Multifactor Contrast Tests. In The 34th Annual ACM Symposium on User Interface Software and Technology (Virtual Event, USA) (UIST ’21). Association for Computing Machinery, New York, NY, USA, 754–768. https://doi.org/10.1145/3472749.3474784

Digital Library

[11]

Euan Freeman, Stephen Brewster, and Vuokko Lantz. 2014. Tactile Feedback for Above-Device Gesture Interfaces: Adding Touch to Touchless Interactions. In Proceedings of the International Conference on Multimodal Interaction - ICMI ’14. ACM, 419–426. https://doi.org/10.1145/2663204.2663280

Digital Library

[12]

Euan Freeman, Stephen Brewster, and Vuokko Lantz. 2016. Do That, There: An Interaction Technique for Addressing In-Air Gesture Systems. In Proceedings of the 34th Annual ACM Conference on Human Factors in Computing Systems - CHI ’16. Association for Computing Machinery, 2319–2331. https://doi.org/10.1145/2858036.2858308

Digital Library

[13]

Euan Freeman, Dong-Bach Vo, and Stephen Brewster. 2019. HaptiGlow: Helping Users Position their Hands for Better Mid-Air Gestures and Ultrasound Haptic Feedback. In 2019 IEEE World Haptics Conference (WHC). IEEE, 289–294. https://doi.org/10.1109/WHC.2019.8816092

[14]

Euan Freeman, Julie Williamson, Sriram Subramanian, and Stephen Brewster. 2018. Point-and-Shake: Selecting from Levitating Object Displays. In Proceedings of the 36th Annual ACM Conference on Human Factors in Computing Systems - CHI ’18. ACM Press, Paper 18. https://doi.org/10.1145/3173574.3173592

Digital Library

[15]

Charles P. Gerba, Adam L. Wuollet, Peter Raisanen, and Gerardo U. Lopez. 2016. Bacterial contamination of computer touch screens. American Journal of Infection Control 44, 3 (2016), 358–360. https://doi.org/10.1016/j.ajic.2015.10.013

[16]

Tovi Grossman and Ravin Balakrishnan. 2005. The Bubble Cursor: Enhancing Target Acquisition by Dynamic Resizing of the Cursor’s Activation Area. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Portland, Oregon, USA) (CHI ’05). Association for Computing Machinery, New York, NY, USA, 281–290. https://doi.org/10.1145/1054972.1055012

Digital Library

[17]

Darren Guinness, Alvin Jude, G. Michael Poor, and Ashley Dover. 2015. Models for Rested Touchless Gestural Interaction. In Proceedings of the 3rd ACM Symposium on Spatial User Interaction (Los Angeles, California, USA) (SUI ’15). Association for Computing Machinery, New York, NY, USA, 34–43. https://doi.org/10.1145/2788940.2788948

Digital Library

[18]

Aakar Gupta, Thomas Pietrzak, Cleon Yau, Nicolas Roussel, and Ravin Balakrishnan. 2017. Summon and select: Rapid interaction with interface controls in mid-air. In Proceedings of the 2017 ACM International Conference on Interactive Surfaces and Spaces - ISS ’17. Association for Computing Machinery, 52–61. https://doi.org/10.1145/3132272.3134120

Digital Library

[19]

Carl Gutwin. 2002. Improving Focus Targeting in Interactive Fisheye Views. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Minneapolis, Minnesota, USA) (CHI ’02). Association for Computing Machinery, New York, NY, USA, 267–274. https://doi.org/10.1145/503376.503424

Digital Library

[20]

Sandra G. Hart and Lowell E. Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research. In Human Mental Workload, Peter A. Hancock and Najmedin Meshkati (Eds.). Advances in Psychology, Vol. 52. North-Holland, 139–183. https://doi.org/10.1016/S0166-4115(08)62386-9

[21]

Morten Hertzum and Kasper Hornbæk. 2007. Input techniques that dynamically change their cursor activation area: A comparison of bubble and cell cursors. International Journal of Human-Computer Studies 65, 10 (2007), 833–851. https://doi.org/10.1016/j.ijhcs.2007.05.001

Digital Library

[22]

Juan David Hincapié-Ramos, Xiang Guo, Paymahn Moghadasian, and Pourang Irani. 2014. Consumed Endurance: A Metric to Quantify Arm Fatigue of Mid-Air Interactions. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Toronto, Ontario, Canada) (CHI ’14). Association for Computing Machinery, New York, NY, USA, 1063–1072. https://doi.org/10.1145/2556288.2557130

Digital Library

[23]

Intel. 2021. Intel RealSense Touchless Control Software. https://www.intelrealsense.com/introduction-to-intel-realsense-touchless-control-software Last Accessed 13/09/2022.

[24]

Mikkel R. Jakobsen, Yvonne Jansen, Sebastian Boring, and Kasper Hornbæk. 2015. Should I Stay or Should I Go? Selecting Between Touch and Mid-Air Gestures for Large-Display Interaction. In Human-Computer Interaction – INTERACT 2015, Julio Abascal, Simone Barbosa, Mirko Fetter, Tom Gross, Philippe Palanque, and Marco Winckler (Eds.). Springer International Publishing, Cham, 455–473.

Digital Library

[25]

Mathias N. Lystbæk, Peter Rosenberg, Ken Pfeuffer, Jens Emil Grønbæk, and Hans Gellersen. 2022. Gaze-Hand Alignment: Combining Eye Gaze and Mid-Air Pointing for Interacting with Menus in Augmented Reality. Proc. ACM Hum.-Comput. Interact. 6, ETRA, Article 145 (may 2022), 18 pages. https://doi.org/10.1145/3530886

Digital Library

[26]

Ville Mäkelä, Jonas Winter, Jasmin Schwab, Michael Koch, and Florian Alt. 2022. Pandemic Displays: Considering Hygiene on Public Touchscreens in the Post-Pandemic Era. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems(CHI ’22). Association for Computing Machinery, New York, NY, USA, 12 pages. https://doi.org/10.1145/3491102.3501937

Digital Library

[27]

Sylvain Malacria, Eric Lecolinet, and Yves Guiard. 2010. Clutch-Free Panning and Integrated Pan-Zoom Control on Touch-Sensitive Surfaces: The Cyclostar Approach. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Atlanta, Georgia, USA) (CHI ’10). Association for Computing Machinery, New York, NY, USA, 2615–2624. https://doi.org/10.1145/1753326.1753724

Digital Library

[28]

Jörg Müller, Robert Walter, Gilles Bailly, Michael Nischt, and Florian Alt. 2012. Looking Glass: A Field Study on Noticing Interactivity of a Shop Window. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems - CHI ’12. ACM Press, 297–306. https://doi.org/10.1145/2207676.2207718

Digital Library

[29]

Jennifer Sarah Pearson, Gavin Bailey, Simon Robinson, Matt Jones, Tom Owen, Chi Zhang, Thomas Reitmaier, Cameron Steer, Anna Carter, Deepak Sahoo, and Dani Kalarikalayil Raju. 2022. Can’t Touch This: Rethinking Public Technology in a COVID-19 Era. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems(CHI ’22). https://doi.org/10.1145/3491102.3501980

Digital Library

[30]

Ken Pfeuffer, Jason Alexander, Ming Ki Chong, and Hans Gellersen. 2014. Gaze-Touch: Combining Gaze with Multi-Touch for Interaction on the Same Surface. In Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology (Honolulu, Hawaii, USA) (UIST ’14). Association for Computing Machinery, New York, NY, USA, 509–518. https://doi.org/10.1145/2642918.2647397

Digital Library

[31]

Hemant Bhaskar Surale, Fabrice Matulic, and Daniel Vogel. 2019. Experimental Analysis of Barehand Mid-Air Mode-Switching Techniques in Virtual Reality. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, USA, 1–14. https://doi.org/10.1145/3290605.3300426

Digital Library

[32]

Ultraleap Ltd.2021. LEGO: Touchless Interactive Play Experience. https://www.ultraleap.com/company/news/case-study/lego-interactive-experience/ Last Accessed 13/09/2022.

[33]

Ultraleap Ltd.2021. Touch free exhibit arrives at the Aquarium of the Pacific. https://www.ultraleap.com/company/news/press-release/touchfree-aquarium-pacific-exhibit/ Last Accessed 13/09/2022.

[34]

Ultraleap Ltd.2021. TouchFree For Kiosks and Digital Signage. https://www.ultraleap.com/enterprise/touchless-experiences/touchfree-solution Last Accessed 11/07/2022.

[35]

Ultraleap Ltd.2021. Touchless Interactive Kiosks: The Air Push Gesture. https://www.ultraleap.com/company/news/blog/touchless-interactions/ Last Accessed 13/09/2022.

[36]

Ultraleap Ltd.2022. PepsiCo innovation: Touch less, smile more!https://www.ultraleap.com/company/news/case-study/pepsico-touchless-ordering/ Last Accessed 13/09/2022.

[37]

Ultraleap Ltd.2022. UST Integrates Ultraleap Tech To Power Faster Checkout Experiences. https://www.ultraleap.com/company/news/press-release/ust-checkout-experience/ Last Accessed 13/09/2022.

[38]

Eduardo Velloso, Marcus Carter, Joshua Newn, Augusto Esteves, Christopher Clarke, and Hans Gellersen. 2017. Motion Correlation: Selecting Objects by Matching Their Movement. ACM Trans. Comput.-Hum. Interact. 24, 3, Article 22 (apr 2017), 35 pages. https://doi.org/10.1145/3064937

Digital Library

[39]

Jo Vermeulen, Kris Luyten, Elise Hoven, van den, and Karin Coninx. 2013. Crossing the bridge over Norman’s gulf of execution : revealing feedforward’s true identity. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems 2013(CHI ’13). Association for Computing Machinery, 1931–1940. https://doi.org/10.1145/2470654.2466255

Digital Library

[40]

Kieran Waugh, Mark McGill, and Euan Freeman. 2022. Push or Pinch? Exploring Slider Control Gestures for Touchless User Interfaces. In Proceedings of the 2022 Nordic Human-Computer Interaction Conference(NordiCHI ’22). Association for Computing Machinery. https://doi.org/10.1145/3546155.3546702

Digital Library

[41]

Kieran Waugh, Mark McGill, and Euan Freeman. 2023. User Study Data for "Proxemic Cursor Interactions for Touchless Widget Control". https://doi.org/10.5281/zenodo.8286402

[42]

Jacob O. Wobbrock, Leah Findlater, Darren Gergle, and James J. Higgins. 2011. The Aligned Rank Transform for Nonparametric Factorial Analyses Using Only Anova Procedures. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Vancouver, BC, Canada) (CHI ’11). Association for Computing Machinery, New York, NY, USA, 143–146. https://doi.org/10.1145/1978942.1978963

Digital Library

[43]

Dennis Wolf, Jan Gugenheimer, Marco Combosch, and Enrico Rukzio. 2020. Understanding the Heisenberg Effect of Spatial Interaction: A Selection Induced Error for Spatially Tracked Input Devices. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–10. https://doi.org/10.1145/3313831.3376876

Digital Library

[44]

Futian Zhang, Sachi Mizobuchi, Wei Zhou, and Edward Lank. 2021. Analyzing Midair Object Pointing Mappings for Smart Display Input. Proc. ACM Hum.-Comput. Interact. 5, ISS, Article 490 (nov 2021), 18 pages. https://doi.org/10.1145/3488535

Digital Library

Cited By

Liu ZLi CLin JXu HXu YNan HCheng WLi JWang B(2024)Advances in the development and application of non-contact intraoperative image access systemsBioMedical Engineering OnLine10.1186/s12938-024-01304-123:1Online publication date: 30-Oct-2024
https://doi.org/10.1186/s12938-024-01304-1
Waugh KMcGill MFreeman E(2023)Demonstrating Proxemic Cursor Input for Touchless DisplaysProceedings of the 2023 ACM Symposium on Spatial User Interaction10.1145/3607822.3618014(1-2)Online publication date: 13-Oct-2023
https://dl.acm.org/doi/10.1145/3607822.3618014

Index Terms

Proxemic Cursor Interactions for Touchless Widget Control
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction techniques
      1. Gestural input

Recommendations

Demonstrating Proxemic Cursor Input for Touchless Displays
SUI '23: Proceedings of the 2023 ACM Symposium on Spatial User Interaction

Touchless gesture interfaces often use cursor-based interactions, where widgets are targeted by a movable cursor and activated with a mid-air gesture. Proxemic cursor interactions are a novel alternative that facilitate faster selection without the need ...
Push or Pinch? Exploring Slider Control Gestures for Touchless User Interfaces
NordiCHI '22: Nordic Human-Computer Interaction Conference

Touchless gesture interfaces enable user interaction without the need for direct physical contact. Recently there has been increased interest in deploying such interfaces over concerns about touchscreen sterility. Many touchless displays use gestures ...
Touchless circular menus: toward an intuitive UI for touchless interactions with large displays
AVI '14: Proceedings of the 2014 International Working Conference on Advanced Visual Interfaces

Researchers are exploring touchless interactions in diverse usage contexts. These include interacting with public displays, where mouse and keyboards are inconvenient, activating kitchen devices without touching them with dirty hands, or supporting ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SUI '23: Proceedings of the 2023 ACM Symposium on Spatial User Interaction

October 2023

505 pages

ISBN:9798400702815

DOI:10.1145/3607822

Editors:
Tony Huang
UTS, Sydney, Australia
,
Misha Sra
UC Santa Barbara, USA
,
Ferran Argelaguet
INRIA, France
,
Pedro Lopes
University of Chicago, USA
,
Mayra Barrera Machuca
Dalhousie University, Canada

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 October 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

SUI '23

Sponsor:

SUI '23: ACM Symposium on Spatial User Interaction

October 13 - 15, 2023

NSW, Sydney, Australia

Acceptance Rates

Overall Acceptance Rate 86 of 279 submissions, 31%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
141
Total Downloads

Downloads (Last 12 months)72
Downloads (Last 6 weeks)7

Reflects downloads up to 16 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Liu ZLi CLin JXu HXu YNan HCheng WLi JWang B(2024)Advances in the development and application of non-contact intraoperative image access systemsBioMedical Engineering OnLine10.1186/s12938-024-01304-123:1Online publication date: 30-Oct-2024
https://doi.org/10.1186/s12938-024-01304-1
Waugh KMcGill MFreeman E(2023)Demonstrating Proxemic Cursor Input for Touchless DisplaysProceedings of the 2023 ACM Symposium on Spatial User Interaction10.1145/3607822.3618014(1-2)Online publication date: 13-Oct-2023
https://dl.acm.org/doi/10.1145/3607822.3618014

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents