Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
research-article

SynchroWatch: One-Handed Synchronous Smartwatch Gestures Using Correlation and Magnetic Sensing

Published: 08 January 2018 Publication History

Abstract

SynchroWatch is a one-handed interaction technique for smartwatches that uses rhythmic correlation between a user's thumb movement and on-screen blinking controls. Our technique uses magnetic sensing to track the synchronous extension and reposition of the thumb, augmented with a passive magnetic ring. The system measures the relative changes in the magnetic field induced by the required thumb movement and uses a time-shifted correlation approach with a reference waveform for detection of synchrony. We evaluated the technique during three distraction tasks with varying degrees of hand and finger movement: active walking, browsing on a computer, and relaxing while watching online videos. Our initial offline results suggest that intentional synchronous gestures can be distinguished from other movement. A second evaluation using a live implementation of the system running on a smartwatch suggests that this technique is viable for gestures used to respond to notifications or issue commands. Finally, we present three demonstration applications that highlight the technique running in real-time on the smartwatch.

Supplementary Material

reyes (reyes.zip)
Supplemental movie, appendix, image and software files for, SynchroWatch: One-Handed Synchronous Smartwatch Gestures Using Correlation and Magnetic Sensing

References

[1]
Daniel Ashbrook, Patrick Baudisch, and Sean White. 2011. Nenya: Subtle and Eyes-free Mobile Input with a Magnetically-tracked Finger Ring. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ‘11). ACM, New York, NY, USA, 2043--2046. https://doi.org/10.1145/1978942.1979238
[2]
Daniel Ashbrook and Thad Starner. 2010. MAGIC: A Motion Gesture Design Tool. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ‘10). ACM, New York, NY, USA, 2159--2168. https://doi.org/10.1145/1753326.1753653
[3]
Daniel L. Ashbrook. 2010. Enabling Mobile Microinteractions. Ph.D. Dissertation. Georgia Institute of Technology, Atlanta, GA, USA. AAI3414437.
[4]
Daniel L. Ashbrook, James R. Clawson, Kent Lyons, Thad E. Starner, and Nirmal Patel. 2008. Quickdraw: The Impact of Mobility and On-body Placement on Device Access Time. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ‘08). ACM, New York, NY, USA, 219--222. https://doi.org/10.1145/1357054.1357092
[5]
Peter Bennett, Stuart Nolan, Ved Uttamchandani, Michael Pages, Kirsten Cater, and Mike Fraser. 2015. Resonant Bits: Harmonic Interaction with Virtual Pendulums. In Proceedings of the Ninth International Conference on Tangible, Embedded, and Embodied Interaction (TEI ‘15). ACM, New York, NY, USA, 49--52. https://doi.org/10.1145/2677199.2680569
[6]
Daniel Boland and Roderick Murray-Smith. 2013. Finding My Beat: Personalised Rhythmic Filtering for Mobile Music Interaction. In Proceedings of the 15th International Conference on Human-computer Interaction with Mobile Devices and Services (MobileHCI ‘13). ACM, New York, NY, USA, 21--30. https://doi.org/10.1145/2493190.2493220
[7]
Agata Brajdic and Robert Harle. 2013. Walk Detection and Step Counting on Unconstrained Smartphones. In Proceedings of the 2013 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp ‘13). ACM, New York, NY, USA, 225--234. https://doi.org/10.1145/2493432.2493449
[8]
Marcus Carter, Eduardo Velloso, John Downs, Abigail Sellen, Kenton O'Hara, and Frank Vetere. 2016. PathSync: Multi-User Gestural Interaction with Touchless Rhythmic Path Mimicry. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI ‘16). ACM, New York, NY, USA, 3415--3427. https://doi.org/10.1145/2858036.2858284
[9]
Ke-Yu Chen, Kent Lyons, Sean White, and Shwetak Patel. 2013. uTrack: 3D Input Using Two Magnetic Sensors. In Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology (UIST ‘13). ACM, New York, NY, USA, 237--244. https://doi.org/10.1145/2501988.2502035
[10]
Christopher Clarke, Alessio Bellino, Augusto Esteves, and Hans Gellersen. 2017. Remote Control by Body Movement in Synchrony with Orbiting Widgets: An Evaluation of TraceMatch. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 1, 3 (Sept. 2017), 45:1--45:22. https://doi.org/10.1145/3130910
[11]
Christopher Clarke, Alessio Bellino, Augusto Esteves, Eduardo Velloso, and Hans Gellersen. 2016. TraceMatch: A Computer Vision Technique for User Input by Tracing of Animated Controls. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp ‘16). ACM, New York, NY, USA, 298--303. https://doi.org/10.1145/2971648.2971714
[12]
Andrew Crossan and Roderick Murray-Smith. 2006. Rhythmic Interaction for Song Filtering on a Mobile Device. In Haptic and Audio Interaction Design (Lecture Notes in Computer Science). Springer, Berlin, Heidelberg, 45--55. https://doi.org/10.1007/11821731_5
[13]
Murtaza Dhuliawala, Juyoung Lee, Junichi Shimizu, Andreas Bulling, Kai Kunze, Thad Starner, and Woontack Woo. 2016. Smooth Eye Movement Interaction Using EOG Glasses. In Proceedings of the 18th ACM International Conference on Multimodal Interaction (ICMI 2016). ACM, New York, NY, USA, 307--311. https://doi.org/10.1145/2993148.2993181
[14]
Augusto Esteves, Eduardo Velloso, Andreas Bulling, and Hans Gellersen. 2015. Orbits: Gaze Interaction for Smart Watches Using Smooth Pursuit Eye Movements. In Proceedings of the 28th Annual ACM Symposium on User Interface Software 8 Technology (UIST ‘15). ACM, New York, NY, USA, 457--466. https://doi.org/10.1145/2807442.2807499
[15]
Emilien Ghomi, Guillaume Faure, StÃľphane Huot, Olivier Chapuis, and Michel Beaudouin-Lafon. 2012. Using Rhythmic Patterns As an Input Method. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ‘12). ACM, New York, NY, USA, 1253--1262. https://doi.org/10.1145/2207676.2208579
[16]
Jun Gong, Xing-Dong Yang, and Pourang Irani. 2016. WristWhirl: One-handed Continuous Smartwatch Input Using Wrist Gestures. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (UIST ‘16). ACM, New York, NY, USA, 861--872. https://doi.org/10.1145/2984511.2984563
[17]
Timo Gotzelmann and Pere-Pau Vazquez. 2015. InclineType: An Accelerometer-based Typing Approach for Smartwatches. In Proceedings of the XVI International Conference on Human Computer Interaction (Interaccion ‘15). ACM, New York, NY, USA, 59:1--59:4. https://doi.org/10.1145/2829875.2829929
[18]
Anhong Guo and Tim Paek. 2016. Exploring Tilt for No-touch, Wrist-only Interactions on Smartwatches. In Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI ‘16). ACM, New York, NY, USA, 17--28. https://doi.org/10.1145/2935334.2935345
[19]
Chris Harrison and Scott E. Hudson. 2009. Abracadabra: Wireless, High-precision, and Unpowered Finger Input for Very Small Mobile Devices. In Proceedings of the 22Nd Annual ACM Symposium on User Interface Software and Technology (UIST ‘09). ACM, New York, NY, USA, 121--124. https://doi.org/10.1145/1622176.1622199
[20]
Ken Hinckley. 2003. Synchronous Gestures for Multiple Persons and Computers. In Proceedings of the 16th Annual ACM Symposium on User Interface Software and Technology (UIST ‘03). ACM, New York, NY, USA, 149--158. https://doi.org/10.1145/964696.964713
[21]
Scott E. Hudson, Chris Harrison, Beverly L. Harrison, and Anthony LaMarca. 2010. Whack gestures: inexact and inattentive interaction with mobile devices. In Proceedings of the fourth international conference on Tangible, embedded, and embodied interaction (TEI ‘10). ACM, New York, NY, USA, 109--112. https://doi.org/10.1145/1709886.1709906
[22]
Sungjae Hwang, Myungwook Ahn, and Kwang-yun Wohn. 2013. MagGetz: Customizable Passive Tangible Controllers on and Around Conventional Mobile Devices. In Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology (UIST ‘13). ACM, New York, NY, USA, 411--416. https://doi.org/10.1145/2501988.2501991
[23]
Hamed Ketabdar, Mehran Roshandel, and Kamer Ali YÃijksel. 2010. Towards Using Embedded Magnetic Field Sensor for Around Mobile Device 3D Interaction. In Proceedings of the 12th International Conference on Human Computer Interaction with Mobile Devices and Services (MobileHCI ‘10). ACM, New York, NY, USA, 153--156. https://doi.org/10.1145/1851600.1851626
[24]
Ju-Whan Kim, Han-Jong Kim, and Tek-Jin Nam. 2016. M.Gesture: An Acceleration-Based Gesture Authoring System on Multiple Handheld and Wearable Devices. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI ‘16). ACM, New York, NY, USA, 2307--2318. https://doi.org/10.1145/2858036.2858358
[25]
Gierad Laput, Robert Xiao, Xiang ‘Anthony’ Chen, Scott E. Hudson, and Chris Harrison. 2014. Skin Buttons: Cheap, Small, Low-powered and Clickable Fixed-icon Laser Projectors. In Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology (UIST ‘14). ACM, New York, NY, USA, 389--394. https://doi.org/10.1145/2642918.2647356
[26]
Gierad Laput, Robert Xiao, and Chris Harrison. 2016. ViBand: High-Fidelity Bio-Acoustic Sensing Using Commodity Smartwatch Accelerometers. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (UIST ‘16). ACM, New York, NY, USA, 321--333. https://doi.org/10.1145/2984511.2984582
[27]
Rong-Hao Liang, Kai-Yin Cheng, Chao-Huai Su, Chien-Ting Weng, Bing-Yu Chen, and De-Nian Yang. 2012. GaussSense: Attachable Stylus Sensing Using Magnetic Sensor Grid. In Proceedings of the 25th Annual ACM Symposium on User Interface Software and Technology (UIST ‘12). ACM, New York, NY, USA, 319--326. https://doi.org/10.1145/2380116.2380157
[28]
Kent Lyons. 2016. 2D Input for Virtual Reality Enclosures with Magnetic Field Sensing. In Proceedings of the 2016 ACM International Symposium on Wearable Computers (ISWC ‘16). ACM, New York, NY, USA, 176--183. https://doi.org/10.1145/2971763.2971787
[29]
Kent Lyons, David Nguyen, Daniel Ashbrook, and Sean White. 2012. Facet: A Multi-segment Wrist Worn System. In Proceedings of the 25th Annual ACM Symposium on User Interface Software and Technology (UIST ‘12). ACM, New York, NY, USA, 123--130. https://doi.org/10.1145/2380116.2380134
[30]
Sylvain Malacria, Eric Lecolinet, and Yves Guiard. 2010. Clutch-free Panning and Integrated Pan-zoom Control on Touch-sensitive Surfaces: The Cyclostar Approach. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ‘10). ACM, New York, NY, USA, 2615--2624. https://doi.org/10.1145/1753326.1753724
[31]
Annie W. Y. Ng and Alan H. S. Chan. 2012. Finger Response Times to Visual, Auditory and Tactile Modality Stimuli. Lecture Notes in Engineering and Computer Science 2196, 1 (March 2012), 1449--1454. https://doaj.org
[32]
Ian Oakley and Doyoung Lee. 2014. Interaction on the Edge: Offset Sensing for Small Devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ‘14). ACM, New York, NY, USA, 169--178. https://doi.org/10.1145/2556288.2557138
[33]
Aman Parnami, Apurva Gupta, Gabriel Reyes, Ramik Sadana, Yang Li, and Gregory D. Abowd. 2016. Mogeste: A Mobile Tool for In-Situ Motion Gesture Design. In Proceedings of the 8th Indian Conference on Human Computer Interaction (IHCI ‘16). ACM, New York, NY, USA, 35--43. https://doi.org/10.1145/3014362.3014365 bibtex: parnami_mogeste:_2016.
[34]
Simon T. Perrault, Eric Lecolinet, James Eagan, and Yves Guiard. 2013. WatchIt: Simple Gestures and Eyes-free Interaction for Wristwatches and Bracelets. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ‘13). ACM, New York, NY, USA, 1451--1460. https://doi.org/10.1145/2470654.2466192
[35]
Gabriel Reyes, Dingtian Zhang, Sarthak Ghosh, Pratik Shah, Jason Wu, Aman Parnami, Bailey Bercik, Thad Starner, Gregory D. Abowd, and W. Keith Edwards. 2016. Whoosh: Non-voice Acoustics for Low-cost, Hands-free, and Rapid Input on Smartwatches. In Proceedings of the 2016 ACM International Symposium on Wearable Computers (ISWC 2016). ACM, New York, NY, USA, 120--127. https://doi.org/10.1145/2971763.2971765
[36]
Jaime Ruiz and Yang Li. 2011. DoubleFlip: A Motion Gesture Delimiter for Mobile Interaction. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ‘11). ACM, New York, NY, USA, 2717--2720. https://doi.org/10.1145/1978942.1979341
[37]
T. Starner. 2013. Project Glass: An Extension of the Self. IEEE Pervasive Computing 12, 2 (April 2013), 14--16. https://doi.org/10.1109/MPRV.2013.35
[38]
Eduardo Velloso, Marcus Carter, Joshua Newn, Augusto Esteves, Christopher Clarke, and Hans Gellersen. 2017. Motion Correlation: Selecting Objects by Matching Their Movement. ACM Trans. Comput.-Hum. Interact. 24, 3 (April 2017), 22:1--22:35. https://doi.org/10.1145/3064937
[39]
David Verweij, Augusto Esteves, Vassilis-Javed Khan, and Saskia Bakker. 2017. WaveTrace: Motion Matching Input Using Wrist-Worn Motion Sensors. In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA ‘17). ACM, New York, NY, USA, 2180--2186. https://doi.org/10.1145/3027063.3053161
[40]
David Verweij, Vassilis-Javed Khan, Augusto Esteves, and Saskia Bakker. 2017. Multi-User Motion Matching Interaction for Interactive Television Using Smartwatches. In Adjunct Publication of the 2017 ACM International Conference on Interactive Experiences for TV and Online Video (TVX ‘17 Adjunct). ACM, New York, NY, USA, 67--68. https://doi.org/10.1145/3084289.3089906
[41]
Melodie Vidal, Andreas Bulling, and Hans Gellersen. 2013. Pursuits: Spontaneous Interaction with Displays Based on Smooth Pursuit Eye Movement and Moving Targets. In Proceedings of the 2013 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp ‘13). ACM, New York, NY, USA, 439--448. https://doi.org/10.1145/2493432.2493477
[42]
Hongyi Wen, Julian Ramos Rojas, and Anind K. Dey. 2016. Serendipity: Finger Gesture Recognition Using an Off-the-Shelf Smartwatch. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI ‘16). ACM, New York, NY, USA, 3847--3851. https://doi.org/10.1145/2858036.2858466
[43]
Haijun Xia, Tovi Grossman, and George Fitzmaurice. 2015. NanoStylus: Enhancing Input on Ultra-Small Displays with a Finger-Mounted Stylus. In Proceedings of the 28th Annual ACM Symposium on User Interface Software 8 Technology (UIST ‘15). ACM, New York, NY, USA, 447--456. https://doi.org/10.1145/2807442.2807500
[44]
Robert Xiao, Gierad Laput, and Chris Harrison. 2014. Expanding the Input Expressivity of Smartwatches with Mechanical Pan, Twist, Tilt and Click. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ‘14). ACM, New York, NY, USA, 193--196. https://doi.org/10.1145/2556288.2557017
[45]
Cheng Zhang, Abdelkareem Bedri, Gabriel Reyes, Bailey Bercik, Omer T. Inan, Thad E. Starner, and Gregory D. Abowd. 2016. TapSkin: Recognizing On-Skin Input for Smartwatches. In Proceedings of the 2016 ACM on Interactive Surfaces and Spaces (ISS ‘16). ACM, New York, NY, USA, 13--22. https://doi.org/10.1145/2992154.2992187

Cited By

View all
  • (2024)picoRing: battery-free rings for subtle thumb-to-index inputProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676365(1-11)Online publication date: 13-Oct-2024
  • (2024)MoiréWidgets: High-Precision, Passive Tangible Interfaces via Moiré EffectProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642734(1-10)Online publication date: 11-May-2024
  • (2024)SoundOrbit: motion-correlation interaction with auditory orbital trajectoriesPersonal and Ubiquitous Computing10.1007/s00779-024-01818-428:5(763-778)Online publication date: 15-Jun-2024
  • Show More Cited By

Index Terms

  1. SynchroWatch: One-Handed Synchronous Smartwatch Gestures Using Correlation and Magnetic Sensing

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
    Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies  Volume 1, Issue 4
    December 2017
    1298 pages
    EISSN:2474-9567
    DOI:10.1145/3178157
    Issue’s Table of Contents
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 08 January 2018
    Accepted: 01 October 2017
    Revised: 01 August 2017
    Received: 01 November 2016
    Published in IMWUT Volume 1, Issue 4

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. interaction techniques
    2. magnetic sensing
    3. one-handed input
    4. smartwatches
    5. synchronous gestures
    6. wearable computing

    Qualifiers

    • Research-article
    • Research
    • Refereed

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)66
    • Downloads (Last 6 weeks)5
    Reflects downloads up to 12 Nov 2024

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)picoRing: battery-free rings for subtle thumb-to-index inputProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676365(1-11)Online publication date: 13-Oct-2024
    • (2024)MoiréWidgets: High-Precision, Passive Tangible Interfaces via Moiré EffectProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642734(1-10)Online publication date: 11-May-2024
    • (2024)SoundOrbit: motion-correlation interaction with auditory orbital trajectoriesPersonal and Ubiquitous Computing10.1007/s00779-024-01818-428:5(763-778)Online publication date: 15-Jun-2024
    • (2023)HeadarProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36109007:3(1-28)Online publication date: 27-Sep-2023
    • (2023)UnifiedSense: Enabling Without-Device Gesture Interactions Using Over-the-shoulder Training Between Redundant Wearable SensorsProceedings of the ACM on Human-Computer Interaction10.1145/36042777:MHCI(1-25)Online publication date: 13-Sep-2023
    • (2023)Robust Finger Interactions with COTS Smartwatches via Unsupervised Siamese AdaptationProceedings of the 36th Annual ACM Symposium on User Interface Software and Technology10.1145/3586183.3606794(1-14)Online publication date: 29-Oct-2023
    • (2023)RetroSphereProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35694796:4(1-36)Online publication date: 11-Jan-2023
    • (2023)Pressure Signal-Based Analysis of Anomalies in Switching Behavior of a Two-Way Directional Control ValveASME Open Journal of Engineering10.1115/1.40564742Online publication date: 11-Jan-2023
    • (2023)GloveSense: A Hand Gesture Recognition System Based on Inductive SensingIEEE Sensors Journal10.1109/JSEN.2023.326235923:9(9210-9219)Online publication date: 1-May-2023
    • (2023)Around-device finger input on commodity smartwatches with learning guidance through discoverabilityInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2023.103105179(103105)Online publication date: Nov-2023
    • Show More Cited By

    View Options

    Get Access

    Login options

    Full Access

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media